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PREFACE 



This book culminates a broad 
inquiry regarding what can be expected 
when questions are asked of mentally 
retarded persons. It started with 
a suggestion by the President's 
Committee on Mental Retardation that 
it might be useful to* conduct a 
periodic national polling of retarded 
citizens of all ages and circum-- 
stances as a means of gathering 
input for policy making purposes. 
The Committee had already commis- 
sioned some work on defining the 
mentally retarded population and 
constituting a representative sample 
of that population. It then 
encouraged the Texas Tech University 
Research and Training Center in 
Mental Retardation to propose a study 
in which the feasibility of inter- 
viewing retarded persons would be 
explored. The study was intended 
to investigate the ability of 
retarded persons to answer questions 
and the reliability and validity 
of their answers; to compare their 
perspectives- with those of the 
nonretarded individuals (e.g.,^ parents) 
who often speak for them; and to 



obtain information about the needs, 
circumstances, and attitudes of 
retarded persons as well as guide- 
lines for constructing interview 
schedules in the futiare. 

The proposal was funded by the 
Rehcibilitation Services Administra- 
tion with the idea that the Center 
would continue to interact with 
PCMR in charting directions. This 
indeed happened, and the President's 
Committee had a great deal of 
influence on the subject matter 
to be included in interview 
schedules as well as the kinds of 
research questions to be addressed. 
We know we did not answer or even 
address all of the issues that 
concerned PCMR, but we' are confident 
that this report will be of value 
to them as well as to a wider range 
of professionals and researchers in 
the developmental disabilities field. 

The authors have many others to thank. 
First, Gerard J.Bensberg, Director 
of the Research and Training Center, 
was indispensable in getting the 
project off the ground and supportive 
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throughout the study- Syng Nam Yough 
and Cheryl Smith contributed to data 
analysis. Staff in the following 
facilities and agencies deserve much 
thanks for their cooperation in making 
children and adults available to us 
for interviewing: the Lubbock State 
School, the Lubbock Public Schools, 
particularly the Special Education 
Department and the Ballenger School; 
Arkansas Mental Retardation- Develop- 
mental Disabilities Services, espe- 
cially the Arkansas Children's Colony 
at Conway; and the Austin Association 
for Retarded Citizens, especially the 
citizen advocacy program and its 
staff. Research of this nature 
always depends on the good will of 
agency and facility staff, from the 
top down, and in this project that 
good will was plentiful. Finally, we 
reserve special thanks for the parents 
and attendants, and most especially 
the retarded people, who actually 
participated ii; our interviews. They 
knew, indeed they were explicitly 
told, that participating would not 



make a difference in their lives, 
would not help them with the problems 
that some of them faced. Yet they 
willingly gave it their best, often 
enjoying the chance to talk about 
themselves and to be listened to care- 
fully. All we can say by way of 
thanks is that we learned more than 
we ever hoped to learn about retarded 
people, as well as about how we "non- 
retarded" people can communicate more 
effectively wi^lh them. Our hope is 
that this report, even though it 
vmcovers many problems in gettinq 
meaningful answers from retarded 
persons will encourage others to 
give them a chance to speak for them- 
. selves. We will show tHat good 
intentions are not enough, that soph- 
istication in phrasing and structuring 
questions and skill in interacting 
with people are needed if the attempt 
is to be fruitful. But as the people 
we talked to have shown us, there is 
much to be gained if we try as hard as 
they do to make the communication 
process work. 
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Chapter 1 



INTRODUCTION 



"If more people do not take a 
more active interest in the life of 
the mentally retarded then we will 
not have any place to go except in 
an institutional living and that is 
not fair because we are God's - 
creatures, to be like you are, 
considered as normal." Unsolicited 
letter cited by Baker, Seltzer and 
Seltze^ (1977). 

The mentally retarded adult who. wrote 
these lines was acting as an effec- 
tive advocate for community residen- 
tial alternatives to institutions 
and for the philosophy of normali- 
zation. As anyone who has worked 
with mentally retarded people knows, 
many of them tell us exactly what is 
on their minds (whether we want to 
know or not). Mentally retarded peo- 
ple, like all people, have khowledge 
of what is going on in their lives, 
their likes and dislikes, and their 
hopes for the future. Historically, 
however, their voices have not been 
heard. Certainly their perspectives 
have rarely been solicited by 



professionals working with them or 
researchers studying them. To be 
sure, there are exceptions. Occasion- 
ally, first-hand accounts of the ex- 
periences of mentally retarded 
persons appear in print (see Stanovich 
and Stanovich, 197 9, for a biblio- 
graphy) . Occasionally "we" even 
solicit their opinions, as illustrat- 
ed by Nirje's (1972) report of a 
Swedish conference of mentally retard- 
ed adults which yielded a set of 
provocative policy recom^nendations.. 
To cite a notable example from the 
research literature, the deinstitu- 
tionalization study by Gollay, 
Freedman, Wyngaarden, and Kurtz (1978) 
not only involved interviews with 
deinstitutionalized persons but also 
included a screening and critique of 
proposed interview materials by a 
group of retarded consvimers. None- 
theless, the predominant pattern has 
been for "us" to speak for "them," 
prompting Stanovich and Stanovich 
(1979) to advocate letting ''them 
talk about us for a change" (p. 83). 
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In partr neglect of U.e perspectives 
of mentally retarded consumers has 
been a result of the models of re- 
tardation that have guided the field. 
When mentally retarded persons are 
viewed as subhuman, as children, or 
as menaces to society, it simply 
follows that their opinions are 
irrelevant and that "we" are the 
ones who know what is and what should 
be happening in their lives. Iff 
instead, we accept the concept that 
they are "God's creatures, to be 
like you are, consider*: ! as normal,'" 
we are motivated to take them 
seriously as human beings with impor- 
tant things to communicate'. Precise- 
ly this concept is gradually infil- 
trating the field of .-nental retarda- 
tion today. The philosophy of 
normalization has had its impact, as 
has the "hroader movement /in society 
to guara-itee the civil rights of \ 
handicapped persons. Groups of 
handicapped persons, serving as their 
own advocates f have had a marked 
impact in seeing that these changes 
have come about. The mentally re- 
tarded, though rarely speaking for 
themselves, have been represented 
by consumer organizations, parents, 
and other representatives. 

One of the major products of the 
handicapped consumer movement has 
been the stipulation in a variety of 
legislation that handicapped clients 
or students be involved, in making 
those decisions which affect them. 
ThuSf rehabilitation agencies must 
insure that a client has agreed to 
an individual rehabilitation plan, 
mental retardation facilities and 
agencies must involve clients in the 
development of habilitation plans, 
and, uinder appropriate circumstances, 
schools must involve not only parents 
but children themselves in the deve- 
lopment of individual education 
programs. These requirements make 
it mandatory that human services 
professionals begin to consider the 



perspectives and preferences of men- 
tally retarded persons. 

Service providers, counselors, and 
teacixers are not the only ones who 
have a stake in obtaining information 
from mentally retarded consumers. 
Researchers, especially those study- 
ing deinstitutionalized and service 
delivery systems, depend on informa- 
tion gathered from mentally retarded 
persons, and indeed increasingly 
express the belief that such persons^ 
should be allowed to speak for them- 
selves, since it is their experiences 
which, are xander study (e.g., Gollay, 
Freedman, Wyngaarden, & Kurtz, 1978). 
Virtually anyone who is involved with 
the mentally retarded has some need 
to understand the perspectives of 
mentally retarded persons and gather 
information from them. 

But how? How does one hear from 
people whose verbal skills, as a 
function of the very natiire of mental 
retardation, are limited? It is with 
this question that the present report 
concerns itself. 

The Nature of the Study 

/ The research was inspired by the 
/president's Committee on Mental Re- 
/ tardation. PCMR became intrigued by 
the idea of conducting a periodic 
poll of a representative national 
sample of mentally retarded persons. 
The proposed survey, to be used with 
persons of all ages and living situ- 
ations , wa.s viewed as a means of 
collecting information about the 
circumstances f needs, and attitudes 
of mentally retarded individuals 
which cculd be used to shape national 
policy. The Research and Training 
Center in Mental Retardiation at Texas 
Tech was approached with the notion 
of conducting a study of the feasibi- 
lity of such a polling, and a pro- 
posal was then written and funded by 
the Rehabilitation Services 
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Administration. It became clear 
that PCMR was interested in a wide, 
range of feasibility issues: how 
to construct a representative 
Scumple, how to access interviewees, g:; 
what to ask, how to ask it, and what 
to do with the resulting information. 
In our own thinking about the proj- 
ect, we decided that the first and 
most important task was to determine 
what can be expected of mentally 
retarded persons in an interview 
situation; whether they can give 
answers, and whether th^ir answers 
are reliable and valid sources of 
information about their circumstances, 
needs, and attitudes. If we assume 
that mentally retarded persons sho . u 
be given opportunities to communicate 
about their own lives and needs, it 
is critical to explore the methodo- / 
logical issues involved in obtaining 
such information. ' , 

In an important sense, then, the 
. study became an exploration of the 
communication skills of the, mentally 
retarded; not their language deve- .. 
lopment, but their ability to under- 
stand questions and use speech to 
accurately convey facts and opinions. 
However, our intent was not just to 
explore deficiencies in communication 
skills among the mentally retarded. 
Comraunication is a two-way process, 
and the ability of mentally ^retarded 
p€:.rsons to respond to a question 
might depend heavily on the form, 
clarity, and salience of the question. 
Thus we sought not only to understand 
the limits of retarded people's 
abilities to participate in inter- 
views but also to identify more and 
less effective ways of asking ques- 
tions of them. 

The four guiding questions of the 
project can be stated as follows: 

1. To what extent can retarded 
piersons respond to questions 
in an appropriate fashion, 
and what factors affect 
their responsiveness? 



2. How reliable are such re- 
sponses, in the sense of 
being consistent over short 
periods of time? 

3. How valid are such responses, 
in the sense of being free 
of systematic biases and 
agreeing with infcriation 
provided by parents? or care- 
takers or documented in 
records? 

4. What types of questions 
appear to optimize respon- 
s ivenes s , re liabi lity , and 
validity? 

Organization of the Report 

The body of this report is based 
on data obtained frpm five different 
interviewing studies. Each study 
involved interviewing a group of men- 
tally retarded children or adults 
about their circiamstahces , needs, 
and attitudes, and also interviewing 
a "significant other" such as a 
parent or attendant to obtain a 
second perspective on each client. 
In Chapter 2 , we briefly review the 
literature relevant ro the project, 
considering both the literature on 
the communication skills of mentally 
retarded persons and the literature 
on the potential for biased responses 
in survey research v/ith the general 
population. In Chapter 3, we de- 
scribe the design of our study, the 
samples interviewed, and the measure- 
ment procedures used so that this 
methodological information can be 
borne in mind in reading later chap- 
ters. In Chapter \4, the logistical 
challenges in conducting interviews 
are considered, and we offer sugges- 
tions based on our experiences for 
carrying out interviews with retarded 
persons. 

Because the five studies were very 
similar in basic purpose and design, 
v/e do not report the findings study 
by study. ^ Inst2ad, the remainder of 

J 



1.3 

11 



the report is organized around the 
four guiding questions posed above. 
In Chapter 5, we draw from the 
various studies to examine respon- o 
siveness, or the extent to which 
mentally retarded persons are able / 
to answer questions. We describe 
the kinds of difficulties inter- 
viewees had in providing an appror 
priate answer; for example, in say- 
ing yes or no to a yes-no question 
or naming something in response to 
an open-ended question. We/look at 
responsiveness as an individual 
characteristic, describing how low 
in the IQ range one can go and ex- 
pect mentally retarded persons to 
be able to respond to simple ques- 
tions, and we relate responsiveness 
to other factors such as the type 
of question asked (yes-no versus 
either-or versus open-ended, and so 
on). . Starting With Chapter 6, we 
move beyond the issue of whether 
or not various questions elicit 
appropriate answers and ask whether 
or not the responses obtained are 
useful. Chapter 6 focuses on the 
issue of reliability by analyzing 
data obtain -.a in two of our studies 
in which institutionalized persons 
were asked the same questions twice, - 
a week apart. If, in an interview- 
ing study, the answers given by a 
person change drastically from week 
to week, the information obtained is 
of little use. Again we examine 
factors (such as question format) that 
affect the extent to which answers 
are stable from week to week. In 
Chapter 7, we turn to the validity 
issue, primarily by examining the 
extent to which answers given by re- 
tarded persons and answers given by 
"significant others" agree. Although 
we do not assume that significant 
others represent "the whole truth," 
we are concerned with the extent to 
which pictures ,of a group provided 
by clients and/ their parents or' 
attendants differ. 



In Chapter 8, we describe direct 
comparisons of alternative ways of 
requesting ^the same information. 
Throughout the five studies, a nurober 
of different comparisons of this type 
were made. In some cases, we com- 
pared alternative question formats 
(e.g., yes-no questions versus either- 
or questions on the same topic) , 
while in other cases we explored the 
effects of alternative question word-, 
ings or alternative probing tsch- 
niques. In each case, we compared 
the questioning strategies at issue 
with reference to three criteria: 
responsiveness, absence of systematic 
response biases, and validity as 
measured by agreement with parents 
or attendants. Thus, we first asked 
which alternative yields appropriate 
responses from a higher proportion 
of the population, assuming that the 
more useful questioning approach is 
the one to which most clients can re- 
spond. Then we compared the two al- 
ternatives in terms of the extent to 
which systematic biases, revealed in 
comparison of a client's responses 
to one format with his or her respon- 
ses to the other, invalidate respon- 
ses. Finally, we determined which 
question is associated with higher 
agreement with the responses given by 
parents or attendants, on the assximp- 
tion that high agreement with 
significant others is an additional 
indicator of validity of responses. 
Of any two approaches to soliciting 
the same information, then, the more 
useful of the two is the approach 
which optimizes responsiveness, does 
not appear to be associated with any 
systematic response biases/ and yields 
answers that concur with those pro- 
vided by significant others. Chapter 
8 thus has many practical implications 
for anyone seeking improved methods 
of obtaining information from men- 
tally retarded persons. 

Finally, in Chapter 9. we attempt 
to summarize the major findings of 
the study and their implicc^tions for 



training mentally retarded persons first to get an overview of the study 

in commxanication skills and for and its findings and then go back /'to 

seeking information from them as a chapters of special interest. / 

service provider, counselor, research- Finally, we have provided a variety 

er, or simply interested party. Some of appendix material of particular 

readers may want to read Chapter 9 interest to those planning interview 

studies, / 
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Chapter 2 



LESSONS FROM THE LiTERATURE 



As in any research project, we 
began our inquiry with reviews of 
previous research, hoping to find a 
body of knowledge on which to base 
our own study. We did not find a lit- 
erature on how best to ask questions 
of retarded persons. As far as we 
can determine, ours is the first such 
study. However, we searched for 
clues in two related bodies of liter- 
ature; the literature on language and 
communication skills of the mentally 
retarded, and the literature on meth- 
odological issues in survey research 
with the general population. This 
chapter overviews what we found most 
pertinent to our research questions 
in these two very different bodies of 
literature. 

Information abrut the communication 
skills of mentally retarded persons 
is extensive, as deficits in lan- 
guage and communication are part of 
the very nature of mental retardation. 
AS experienced mental retardation re-^ - 
searchers, we of course knew that the 
limited verbal skills/ of. many retard- 
ed persons would limit the...kinds of 



questions that could be asked of them. We 
kr\ew that verbal interviewing would 
be all but impossible with profound- 
ly retarded persons, but were not as 
sure how much could be expected of 
those at higher levels of retarda- 
tion. We hoped to gather informa- 
tion about whether we could expect 
of a retarded person essentially what 
could be expected of a normal child 
of the same mental age, or whether 
some of the language and communica- 
tion difficulties of retarded per- 
sons are not so much a problem of 
developmental delay cxs a matter of 
qualitative difference. For example, 
if retarded persons have been f jund 
to have serious difficulties in un- 
derstanding quantitative concepts 
and terms, beyond those that. could be 
predicted on the basis'^of mental age, 
we would want to avoid their use in 
interviews. 

Reviewing this vast literature was 
-difficult, for participating in an 
interview is an activity that calls, 
on all of a person's language capac- 
ities. Receptive skills are needed 
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to linderstand the question being 
■asked; expressive skills to give an 
knswer. Moreover, reception and ex- 
pression, or comprehension and repro- 
duction,, must be applied to all of 
the three subsystems of language: 
sound\ meaning, or semantics, and 
grammar or syntax. Thus, we had to 
be concerned with hearing deficits 
and speeX^h or articulatory problems 
among retarded .persons ; we had to ex- 
plore theii^ vocabulary and comprehen- 
sion of various concepts; and we had 
to consider their ability to under- 
stand the strticture of questions 
posed to them as well as to structure 
answers. \ 

The selective literature review we 
present here considers in turn the 
sound, meaning, and'>grammatical sys- 
tems of language and^suggests what 
can be expected of retarded persons • 
in interview situations'. However, 
this body of literature '^cjid little 
to tell us whether the answers of 
retarded persons can be expected to 
provide meaningful insighr into their 
lives. We can ask a simple yes-no 
question, for example, and a r'^tarded 
person may appear to understands the 
question and answer "yes." But \s 
that "yes" a reflection of reality? 
Would the person's parent or soneorie 
else who knows the person well con- \ 
cur? These are issues which have not 
been addressed in the mental retarda- ' 
tion field; yet they are critical if 
we are concerned not just with lin- 
guistic competencies but with the use 
of those competencies for a purpose: 
to communicate accurately one's needs, 
circumstances, and attitudes. 

Fortunately, we had another body of 
literature to turn to for clues con- 
cerning the issue of response valid- 
ity. Researchers who use survey re- 
search methods with the general popula- 
tion have long been ccncerned with threats 
to response validity . They have repeat- 
edly found, for example, that many people 



give biased answers to put themselves 
in a favorable light and that how one 
structures and phrases a question may 
influence what answers are given. We 
assumed that if certain biasing effects 
have been uncovered in survey research 
with the general population, there is 
no reason to believe that they will 
not surface in interviews with re- 
tarded persons. On the contrary, as 
we will show, there is reason to be- 
lieve that threats to response vali- 
dity will be even more acute in inter- 
views of retarded persons than they 
are in interviews of nonretarded per- 
sons. Thus this ch^^ter's second sec- 
tion will overview what survey research- 
ers have discovered about response bias. 

Language and Communication 
Among Retarded Persons 

Speech and communication problems 
are generally viewed as characteris- 
tic of mentally retarded persons 
(Blount, 1968; Karlin & Strazzula, 
1952; Keane, 1972), and have frequent- 
ly been used as important diagnostic 
indicators of mental retardation 
(Blount, 1969; Matthews, 1971). Un- 
fortunately, research has provided 
little consistent normative data on 
the general level of linguistic com- 
petence to be expected in retarded 
persons of a particular intelligence 
.level (Matthews, 1971). This lack of 
vgeneral expectations about language 
Competencies can be attributed to the 
va^vied approaches used to study lan- 
guage development in the retarded 
(Yoder & Miller, 1972). Studies are • 
difficult to compare directly due to 
methodological differences , variations 
in the populations studied, and dif- 
ferences in the specif ic types of lan- 
guage behavior measured. However, 
those who have surveyed the literature 
of language development of the mentally 
retarded have been able to draw some 
broad conclusions. 
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LANGUAGE AND INTELLECTUAL 
FUNCTIONING 

The retarded child typically ac- 
quires language and speech consid- 
erably later than the nonretarded 
child <Baroff, 1974; Cromer, 1974; 
Matthews, 1971; Yoder & Miller, 
1972) . While many developmental 
sequences are delayed for retarded 
children, the areas of speech and 
higher intellectual functions shov; 
the greatest amoi:.nt of delay (Karlin 
& Strazzula, 1952). For example, 
Schlanger (1953a) reported that the 
average age of onset of speech in a 
group of institutionalized retarded 
children was slightly over three 
years, while nonretarded children 
begin speaking about the first year 
(McCarthy, 1954). However, even 
though retarded children are reported 
to lag behind their nonretarded peers 
on such measures as sentence length, 
sentence complexity, discrimination 
of speech sounds, and percentage of 
nouns used (Spreen, 1965), most 
authors agree that the course of 
language development generally 
parallels that of normal children - 
That is, it is not qualitatively' 
different (Lackner, 1976; Lenneberg, 
Nichols, & Rosenberg, 1964; Yoder & 
Miller, 1972) . 

Baroff (1974) has attempted to pro- 
vide some rough escpectations re- 
garding the lingijxstic competence 
of retarded children and adults as 
a fxinction of mental age and degree 
of retardation. As shown in the svim- 
mary of these norms (Table 2.1), 
Baroff suggests that the major 
language deficit for retarded persons 
involves expressive language abili- 
ties. As has been found true of 
normal children, retarded persons 
are usually able to understand more 
speech than they are ehle to produce. 
Therefore, a general strategy for 
interviewing the retarded might be 
to devise questions which minimize 
productive demands (e.g., through 
simple yes-no questions , or forced- 



choice questions, or even questions 
that require only pointing) . 

Baroff 's norms for retarded children 
also suggest that the order of lan- 
guage acquisition follows the devel- 
opmental sequence reported for nonre- 
tarded children (McCarthy, 1965). 
For our purposes, the most useful 
information that emerges from 
Baroff 's data is that the ability to 
respond to questions is not evident 
prior to mental age four or five. 
Therefore, efforts directed at inter- 
vievdng retarded persons below this 
mental age may be inadvisable and of 
limited value. 

Baroff also emphasizes that speech 
problems and language delays are more 
pronounced at the lower levels of in- 
tellectual functioning, and that 
speech development in the retarded 
child is closely associated with men- 
tal age. For the most part, research 
on the language development of the 
retarded supports these observations 
(Karlin & Strazzula, 1952; Lyle , 
1961b; Sirkin & Lyons, 1941). 

As a general rule, then, the higher 
the level of intelligence, the better 
the language ability of the retarded 
individual (Keane, 1972). Many early 
studies indicated that speech and 
language measures are positively cor- 
related with intellectual level in 
retarded populations , even though 
these correlations are often of only 
a moderate degree. For excimple, 
Spradlin (1963) reviewed four studies 
which correlated articulation and 
intelligence measures for institu- 
tional populations and found that 
the coefficients ranged from .41 to 
.58. Schlanger (1958) found that 
auditory word discrimination corre- 
lated .59 with mental age, and other 
studies have reported correlations 
ranging from .42 to .76 between vo- 
cabulary size and intelligence 
(Condell, 1959; Mein, 1962; Sloan & 
Bensberg, 1954). By contrast, 
chronological age has little or no 



Table 2.1; Summary of language norms for retarded children and adults as a function of mental 
age, chronological age, and degree of retardation from Baroff (1974) 



Mental 
Age 



yr 



1-2 yrs 



2-3 yrs 



4-5 yrs 



5-6 yrs 



6-7 yrs 



7-11 yrs 



Chronological 
Age 



4+ 



3-6 
8+ 



3i-5 

5- 7 

6- 9 
10+ 



6-9 
10-12 
12-15 
16+ 



8-10 
11-13 
13-15 



10-12 
14+ 



years 



years 
years 



years 
years 
years 
years 



years 
years 
years 
years 



years 
years 
years 



years 
years 



13-15 years 



Degree of 
Retardation 



Profound 



Severe 



Mild 

Moderate 

Severe 

Profound 



Mild 

Moderate 

Severe 

Profound 



Mild 

Moderate 
Severe 



Mild 

Moderate 



Mild 



Language Behavior 



Expressive: Imitates sounds. Laughs or smiles 
in response. May say "mama" or 
"dada". At a pre-speech level: 
crying, vocalization, and gestures 

Receptive: Some undeL'standing of gestures and 
very familiar words (e.g., "no"). 



Expressive: May use one or two words, but pre- 
speech forms continue to predomi- 
nate. 



Expressive: May have considerable speaking 

vocabulary though speech will be 
particularly impaired in children 
with more than mild retardation. 
If nonverbal, as in severely and 
profoundly retarded; there may be 
use of gestures to communicate. 

Receptive: Understand simple verbal communi- 
cations, (e.g., following direc- 
tions, responding to questions). 

Expressive: May have considerable speaking 

vocabulary though speech will be 
particularly impaired in children 
with more than mild retardation - 
If nonverbal, as in severely and 
profoundly retarded, there may be 
use of gestures to communicate. 

Receptive: Understands simple verbal communi- 
cations, (e.g., following direc- 
tions, responding to questions). 



Expressive: Mildly retarded child may be using 
fairly normal sentence structure 
and have speech of good intelligi- 
bility. At more severe levels of 
retardation, language may be at 
the phrase or single-word level and 
also be indistinct in pronunciation 



Expressive: May be able to carry on a simple 
conversation and use complex sen- 
tences (more true of mildly than 
moderately retarded children). 



Expressive: Essentially normal fluency though 
pronunciation problems persist. 
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Table 2.1: continued 



Mental 
Age 


IQ 
Level 


Degree of 
Retardation 


Language Behavior 




8-11 yrs 


52-68 


Mild 


Language is adequate for ordinary purpose of 
communication at both expressive and receptive 
levels. 




6-8 yrs 


36-51 


Moderate 


Language is generally functional for purposes 
of communication, but intelligibility of speech 
may bo much impaired. 




4-6 yrs 


20-35 


Severe 


Understanding language i? likely to be much 
better than the ability to express it. Speech 
may be very poorly articulated and difficult to 
understand. 




0-4 ' yrs 


0-19 


Profound 


Ability to understand is likely to far exceed 
ability to speak. There may be little or no 
speech. Capable of following simple directions. 

» 

/ 
/ 

/ 
1 

' ! 

/ 

i 

1 
1 

* 

— ^ ■ 
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relationship to linguistic ability 
in the retarded child (Bangs, 1942; 
Schlanger, 1953b). The' relationship 
between intelligence and language 
generally holds for persons with a 
mental age of eight years or less, 
at which point the degree of corre- 
lation diminished (Spreen, 1965). 
It is interesting to note that Goda 
and Griffith (1962) have reported 
similar results for nonretarded 
children. They contend that speech 
development is directly related to 
intell^^ntual growth in normal chil- 
dren 1 age eight. After this 
point, further development is minimal 
and consists mainly of mastering more 
complex sentence structures and 
gradually increasing vocabulary size. 
Therefore, prior to mental age eight, 
one would expect to find language 
acquisition and proficiency closely 
related to intellectual level in both 
retarded and nonretarded populations. 
The language development of the re- 
tarded child would be very similar 
to that of the nbhretarded child of 
an equivalent mental age. Conse- 
quently, retarded persons below men- 
tal age eight would not have mastered 
the fundamental aspects of language 
and would evidence more immature 
speech forms than nonretarded persons 
of the same chronological age. 

While most authors report that speech 
impairments and language deficits are 
more numerous and severe in retarded 
populations, ?.t is generally accepted 
that no specific types of language 
disorders are found in the speech of 
the retarded that are not also found 
in nonretarded populations (Keane, 
1972; Matthews, 1971; Spreen, 1965). 
Similarly, lienneberg and his asso- 
ciates (1967; Lenneberg, Nichols, & 
Rosenberg, 1964) have concluded that 
no structures are found in the lan- 
guag'2 of the mentally retarded that 
do not occur in the course of devel- 
opment for nonretarded children. 
These authors contend that differ- 
ences observed in the speech and 



language patterns of the retarded are 
primarily quantitative in nature and 
do not represent qualitative language 
differences . 

In contrast to previous research, how- 
ever, Lenneberg et al. (1964) reported 
that chronological age was a better . 
predictor of language achievement than 
intelligence in a three-year study of 
Down's Syndrome children whose ages 
ranged from 3 to 22. They also con- 
cluded that language acquisition in 
Down's Syndrome children depends more 
on physiological maturation than on 
cognitive development . Lennebe?rg ' s 
findings highlight two important con- 
siderations. First, etiology plays a 
role in determining the course of 
speech development and competence in 
the retarded (Cromer, 1974; Spreen, 
1965; Webb & Kinde, 1967). Therefore, 
it would seem necessary to distinguish 
type of mental retardation as well as 
degree of retardation when assessing 
the linguistic ability of the re- 
tarded. Secondly, re search has sug- 
gested that factors other than cog- 
nitive ability may influence language 
acquisition, and therefore we must 
recognize how 'such factors affect the 
language behavior of the retarded. 

LANGUAGE AND ETIOLOGY 

Early research designed to identify 
specific language deficits among the 
various diagnostic categories of men- 
tal retardation has typically compared 
Down's Syndrome or' brain-damaged 
groups with other groups of mentally 
retarded persons. In general, brain- 
damaged retarded children show the 
greatest amount of unevenness in their 
expressive and receptive language 
abilities (Rigrodsky & Goo 1951) . 
For example , when compared to non- 
brai,n-damaged groups, they are inferi- 
or oh auditory discrimination and 
perception tasks (Spreen, 1965; 
Schlanger, 1958), and are inferior in 
making verbal associations, integrat- 
ing verbal concepts, and using 
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visual -motor perceptual abilities in 
speech production (Gallagher, 1957). 
On the other hand, the brain-damaged 
as a group typically have larger vo- 
cabularies , use more advanced modes 
of verbal expression, give more de- 
tailed and complete definitions for 
words (Bijou & Werner, 1944, 1945), 
and are better on verbal imitation 
and speech production tasks 
(Gallagher, 1957). 

In contrast, Down's Syndrome children 
as a whole are consistently poorer on 
most language measures when compared 
to other groups of retarded children 
(Lyle, 1960c, 1961a, b) . Mein (1961) 
has noted that Down's Syndrome chil- 
dren decrease their use of nouns at 
a later age than do most other groups 
of retarded children, which has been 
interpreted as representing a more 
concrete and immature style of think- 
ing. Speech defects' are also reported 
to occur more frequently among Down's 
Syndrome children (Schlanger & 
Gottsleben, 1957) . In particular, 
several authors have noted a greater 
incidence of stuttering among Down's 
Syndrome groups (Gottsleben, 1955; 
Schaeffer & Shearer, 1968; Schlanger, 
1953a). Finally, Bilovsky and Share 
(1965) suggest that a deficit in 
auditory-sequential memory is a major 
factor in the language retardation of 
Down's Syndrome children. 

Many of these findings have been sup- 
ported and extended in a recent inves- 
tigation by Rohr and Burr. (1978). 
They compared four diagnostic groups 
of retarded subjects (Down's Syndrome, 
biological brain damaged, environmen- 
tally caused, and unknown etiology) 
according to their performance on the 
Illinois Test of Psycholinguistic 
Abilities. All four groups exhibited 
a similar pattern of scores on the 
subtests, doing best in verbal expres- 
sion and worst in auditory memory. 
However, it was found that the Down's 
Syndrome group consistently scored 
lower than the other three groups on 



all auditory-verbal subtests except 
verbal expression. In contrast, the 
environmentally-caused retardation 
group scored higher on all subtests 
than did the other diagnostic groups, 
although the differences were not 
always significant. The scores for 
the other two diagnostic groups were 
more variable, but generally fell be- 
tween the extremes defined by the 
Down's Syndrome group and the environ- 
mentally-caused retarded group. The 
authors suggest that an auditory mem- 
ory deficit contributes to the lan- 
guage problems of all retarded groups, 
with Down's Syndrome children si^mply 
being affected the most rather than 
being unique as Bilovsky and Share 
(1965) implied. Therefore, etiology 
appears to have an overall influence 
on the degree of auditory-verbal de- 
ficiencies in the mentally retarded, 
but no specific pattern of language 
disability can be linked to a specific 
type of retardation. 

LANGUAGE AND OTHER FACTORS 

Matthews (1971) suggests that there 
may be many other explanations for 
poor speech development and language 
competence besides low intellectual 
functioning. For example, emotional 
disturbances, hearing loss, and lack 
of verbal stimulation may delay lan- 
guage acquisition or result in abnor- 
mal speech patterns. 

Environmental deprivation is frequent- 
ly cited as producing negative effects 
on the language development of chil- 
dren, and the adverse effects of in- 
stitutionalization on language devel- 
opment has been studied extensively 
in both retarded and nonretarded pop- 
ulations (Spradlin , 1963 ; Spreen , 1965 ; 
Spitz, 1945, 1946; Skeels', Udegraff, 
Wellman, & Williams, 1938). Institu- 
tional living has been associated with 
lower scores on vocabulary measures 
in both normal (Skeels et al. 1938) and 
retarded children (Little & Williams, 
1^37). Schlanger (1954) reported 
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that institutionalized retarded 
children produce shorter sentences t 
while Sievers and Essa (1961) found 
that institutionalized children 
scored lower on the Developmental 
Language Facilities Test than did 
a matched group of noninsMtution- 
alized children. Finally, Lyle 
(1959, 1960a, b, 1961b) has con- 
eluded from a series of studies on 
the effects of institutionaliza- 
tion that noninstitutionalized re- 
tarded children are superior on 
overall language ability and that 
the negative effects of institu- 
tionalization may be greater at 
certain points in the language 
development sequence. These 
authors point to the effects of 
family separation, emotional de- 
privation , lack of good speech • 
models , and lack of speech moti- 
vation as the causes of these def- 
icits. 

More recent research has sought to 
determine if some language func- 
tions are more affected by insti- 
tutionalization than others are . 
For example , Montegue , Hutchinson , 
and Matson (1975) analyzed psycho- 
logical and sociological language 
content categories and found little 
difference between language samples 
of. institutionalized and noninsti- 
tutionalized retarded children on 
these measures. They suggest that 
institutionalization may produce ' 
the same negative effects on lan- 
guage content as were reported pre- 
viously for measures of language 
structure and vocabulary. 

More recently, McNutt and Leri 
(19/^) examined Both the structure 
of spoken language and specific 
• auditory-verbal abilities as meas- 
ured by the Illinois Test of 
Psycholinguistic Abilities among 
institutionalized and noninsti- 
tutiona] ized retarded. children. 



Their findings supported the 
suggestion of Montegue et al. (1975) 
that institutionalization may have 
selective effects on the language 
abilities of the retarded- In 
particular, they found that their 
institutionalized groups had sig- 
nificantly poorer scores on audi- 
tory reception, verbal expression, 
and auditory closure. However, in 
^ontrast to previous research, they 
"found no differences between the 
two groups on measures of language 
structure. Overall, then, when 
differences are found between groups 
of institutionalized and noninsti- 
tutionalized retarded persons, they 
generally favor those who do not 
live in institutions , but more 
markedly in some areas of competence 
'than in others. 

In addition to examining the effects 
of institutionalization, current re- 
search has also looked at differences 
in the early mother-child verbal 
interactions of retarded and non- 
retarded infants . ■ ^bal interac- 
tions between retarc»^a children 
and their mothers appear to be 
different from the communication 
patterns of honretarded children and 
their mothers (Buixim, Rynders , & 
Tumure, 1974; Gutmann & Rondal, 
1979; Marshall, Hegrenes, & Goldstein, 
1973), For example, Buium et al. 
(1974) found .that mothers- of Down's 
Syndrome children appear to be 
modeling shorter and grammatically 
,less complex verbalizations for 
their retarded children than mothers 
of nonretarded children do. Current- 
ly, we do not know whether such 
verbal modifications made by the 
mothers of retarded children ad-- 
versely affect their children's 
language development • Gutmann and 
Rondal (1979) have suggested that 
these adaptations may not be func- 
tionally inappropriate to the re- 
tarded child's language comprehension 
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needs; however, they may contribute 
to the simple language structure 
generally reported to occur in the 
speech of the mentally retarded. 

In summary, then, speech and com- 
munication problems abound among 
retarded persons, with the major 
difficulties being evident in the 
expressive area. Even though 
language development appears to be 
delayed in the majority of re- 
tarded children, the sequence of 
development has been found to 
para^llel that of nonretarded chil- 
dren and is not considered to be 
qualitatively different. A lack of 
general language expectations pre-' 
vents us from making accurate pre- 
dictions as to what can be expected 
of the retarded person of a par- 
ticular mental age; however, on the 
average, language competence in- 
creases and speech problems de- 
crease with higher intellectual 
functioning. In addition, a nxiiriber 
of other factors such as environ- 
ment may contribute to the lan- 
guage problems of the retarded and 
produce more severe language def- 
icits than would be expected from 
their level of intellectual func- 
tioning. We will now focus on 
more specific speech and language 
deficits among the retarded that 
should be taken into account in 
interviewing them*. 

PROBLEMS RELATED TO SPEECH AND 
HEARING DEFECTS 

Research on the prevalence of 
speech and hearing disorders among 
the menially retarded has been re- 
viewed extensively by several 
authors (Bensberg & Sigelman, 1976; 
Keane, 1972; Matthews, 1971; 
' Spradlin, 1963; Spreen, 1965). 
There is substantial evidence that 
speech and hearing defects are more 
frequent in mentally retarded pop- 
ulations than in the general pop- 



ulation. However, wide variation 
among* prevalence figures is also 
evident. 

Speech Defects 

Speech defects among retarded 
interviewees may make it difficult 
for them to express themselves and 
for interviewers to understand them. 
Spradlin (1963) reviewed 14 studies 
which surveyed the prevalence of 
speech defects in mentally retarded 
children and concluded that the per- 
centage of institutionalized re- 
tarded persons having speech defects 
varies from 57 to 72 percent.* 
Matthews' (1971) review of the lit- 
erature reported figures which varied 
from 18 to 94 percent, with median 
rate. of 56 percent. Thirteen of the 
18 retarded samples studied were also 
from institutional settings. 

According to Spreen (1965) , the pre- 
valence of speech defects varies as 
a function of intelligence within 
the retarded population. He esti- 
mates that almost 45 percent of those 
classified as mildly retarded exper- 
ience speech problems of some kind. 
This figure increases to 90 percent 
in the moderately retarded range , 
and nearly all children in the 
severely and profoundly retarded 
ranges demonstrate some kind of 
communication disorder, which may 
include lack of speech. 

Those authors who have attempted to 
assess. the frequency with which 
specific speech problems occur cimong 
the mentally retarded tend to agree 
that articulation and voice disorders 
constitute the largest percentage 
of speech difficulties (Keane, 1972; 
Spradlin, 1963). -There is evidence 
to indicate that this is also true 
in nonretarded populations (Spradlin, 
1963) andvthat, in general f the types 
of speech problems found in retarded 
groups are also found in normal 
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populations (K.eane, 1972; Sprad- 
lin, 1963; Spreen, 1965). 

Stuttering is another speech diffi- 
culty which, like articulation and 
voice disorders , seems to be es- 
pecially commoa among the retarded. 
Keane (1972) rates stuttering as the 
third most common speech defect of 
the mentally retarded, but noted 
that the higher frequency of 
^^tttfetering in this group is still 
controversial. For excxmple^ Karlin 
and Str/fzzula (1952) ob:5erved only 
one case of stuttering out of 50 
mentally retarded children f a find- 
ing which corresponds with the 
prevalence figures typically cited 
for the general population (2%) . 
There seems to be more agreement 
that a higher ^ than normal incidence 
of stuttering exists among persons 
with Down's Syndrome (Shubert, 1966); 
Schlanger & Gottsleben, 1957; 
Gottsleben, 1955). In addition, 
voice disorders such as low husky 
voices are reported to be especially 
common among Down's Syndrome- children 
(Benda, 1946; Karlin & Strazzula, 
1952; West, Kennedy, & Carr, 1947), 

Mentally retarded persons have also 
demonstrated difficulty in making 
sound discriminations, which con- 
tribute? to poor speech production. 
In one study, Schlanger (1953b) de- 
termined that only 19 percent of his 
sample of retarded children ages 8 
to 16 years were able to make sound 
discriminations at the second grade 
level of proficiency, even though 
the average mental age for the sample 
was 6-8. In a previous investiga- 
tion, Schlanger (1953a) reported 
that these sam^ children were also 
inferior to nonretarded children 
matched on mental age in auditory 

memory span for vowels.. 

.1 

The development of speech sounds 
and speech proficiency in retarded 
children has been examined in sev- 



eral studies. Matthews (1971) cites 
a 1942 study b^' Bangs as one of the 
most systematic studies of typical 
errors found in the speech of the 
mentally retarded. Bangs. analyzed 
the types of verbal substitutions, 
omissions, and additions made by 53 
retarded children. He found that 
mental age was more closely related 
to speech proficiency as measured by 
the number of articulation erro::s 
(r = .41) than either intelligence 
(r = .33) or chronological age 
(r = -.11). In general, the types 
of articulatory errors made by re- 
tarded children were very similar 
to those produced by nonretarded 
children of a comparable mental age. 

During the same year , Irwin (1942) 
reported on the speech sound devel- 
opment of ten mentally retarded chil- 
drer> ranging from one to five ^ears 
old over a one-year period. Unlike 
Bangs, he concluded that speech 
sound development in his sample of 
retarded children was qualitatively 
different from that of normal chil- 
dren, because their level of pro- 
ficiency approximated that of nor- 
mal children less than one-year old. 
However, Irwin's findings may ac-^ 
tvislly indicate that speech sound 
acquisition was severely delayed 
in this group" rather than being 
different from that reported for 
normal children because his sample 
was' still quite young. 

A more extensive study was reported 
by Karlin and Strazzula (1952), 
using subjects with a wider range of 
intelligence scores and ages than 
those studied by Irwin. The authors 
observed that the majority of chil- 
dren had speech defects which were 
similar to those found in normal 
children, but that the defects were 
more prevalent and often more severe . 
Similarly, Schlanger 's (1953a, 1953b) 
data. also point to a delay of normal 
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speech development in retarded 
childreri, rather than the develop- 
ment of deviant speech patterns, and 
support Bangs' rather than Irwin's 
findings- Finally, Goda and Griffith 
(1962) found that both intelligence 
and mental age were rela-^ed to 
articulation scores in a i^ample of 
retarded adolescents (r = .58 and 
.52, respectively). 

Therefore , when interviewing the 
mentally retarded one would expect 
to find the same types of speech 
defects commonly found in normal 
children, but with greater frequency 
and often greater severity. The 
evidence seems to suggest that 
speech proficiency is related to the 
retarded individual's level of 
intelligence n and one could expect 
the speech capacity of the retarded 
person to correspond to that of a 
nonretarded child of a comparable 
mental age. Several authors have 
indicated that articulation and 
voice disorders are the most common 
types of speech difficulties among 
the retarded. This would suggest ; 
that interview techniques whi.ch / 
make minimal expressive demands on 
the interviewee would be most 
effective with mentally retarded 
persons • 

Hearing Disorders 

Hearing disorders not only 
interfere with language development 
but obviously make for difficulties 
in understanding questions ashed ^ , 
in an interview. Like the liter- / 
ature on speech disorders , the j 
literature on hearing disorders , 
cimong the mentally retarded suggests 
wide variation among the estimated 
percentages of retarded persons 
with hearing impairment. However, 
the estimates have consistently 
exceeded the five percent pre- 
valence figvire commonly cited for 
hearing losses in normal public 



school children. Individual 
studies have reported estimated per- 
centages of hearing problems in 
retarded populations ranging from 
two-and-a-half to eighteen times 
that found in the general population 
(Birch & Matthews, 1951; Rittmanic, 
1959; Schlanger & Gottselben, 1956). 
Therefore , despite conflicting 
figures, the consensus is 
that the < ccurrence of hearing 
problems is considerably greater 
in this population t^ian in nonre- 
tarded populations (Bensberg & 
Sigelman, 1976; Matthews, 1971; Lloyd 
& Reid, 1967) . 

One major consideration in deter- 
mining the prevalence of hearing 
disorders in any population involves 
the choice of criteria for defining 
a^ hearing loss. Several authors 
have noted that variation in hearing 
loss criteria has created problems 
in comparing prevalence figures 
(Webb & Kinde, 1967; Kodman, 1958; 
Lloyd & Reid, 1967). Kodman (1958) 
reviewed seven studies on the pre- 
valence of hearing problems in the 
retarded and found that the rate of 
hearing impairments ranged from 13 
to 49 percent. He proposed that the 
criteria for defining- a hearing prob- 
lem should be standardized to re- 
present a loss of 30 decibels or 
more in either ear at one or mo.re 
frequency (rsmge 1^5 to 8000 cycles 
per second) , on th6 basis that most 
hearing losses below this level are 
not severe enough /to interfere with 
speech reception and language acqui- 
sition. Accordir^g to these criteria, 
he estimated that the prevalence of 
hearing losses Among the mentally 
retarded would/ then average 15 to 
20 percent. While Lloyd (1970) 
agreed with these estimates, he 
argued that^a less stringent crite- 
rion for hearing loss is more appro- 
priate. 
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Most studies which have examined the 
severity of hearing disorders at 
various levels of retardation have 
fo\ind almost no relationship between 
the percentages of hearing problems 
observed and the level of measured 
intelligence (Reynolds & Reynolds^ 
1979; ^igrodsky, Prunty, & Glovsky^ 
1961; Schlanger & Gortsleben, 1956; 
and Siegenthaler & Krzywicki, 1959). 
However/ this may be an artifact, 
since severely and profoundly re- 
tarded persons- are often untestable 
by audiometric screening (Hogan, 
1973). In addition, hearing dis- 
orders are known to increase with 
age in all populations, and there 
are several studies which demon- 
strate this fact in retarded pop- 
ulations (Kodman, 1958; Kodman, 
Powers^ Philip, & Weller, 1958; 
Schlanger & Gottsleben, 1956). 
Schlanger and Gottsleben (1956) 
observed a 25.7 percent hearing loss 
rate in their under 20 age group, 
but this percentage increased to 
41.4 percent in their over 20 age 
group. Kodman et al. (1958) used 
the same hearing loss criteria and 
obtained more conservative figures. 
Their young group (mean age = 15.4 
years) demonstrated a 19.0 percent 
rate of hearing loss, while their 
older group (mean age =38.7 years) 
showed a 23.^ percent hearing loss 
rate . 

Even though there is considerable 
variation among the surveys of hear- 
ing problems in the retarded, on the 
average, hearing impairments seem to 
occur four to five times more often 
in retarded children than, in normal 
children. Studies which use less 
stringent criteria to define a hear- 
ing loss and which evaluate older 
subjects frequently show higher prev- 
alence figures. . Finally although 
there is no solid research evidence, 
one must consider the possibility 
that a hearing loss in a retarded 



child is more handicapping than the 
same loss in a normal child. Lloy<i 
and Cox (1972) have suggested that 
"the retarded do not demonstrate the 
same adaptive behavior or capacity 
to compensate for sensory involve- 
ment such as auditory or visual 
losses as do nonretarded individuals" 
(p. 22). Possibly then, a hearing 
handicap of even minor severity could 
represent a serious disability to the 
retarded person and decrease his or 
her. ability to learn about the world 
and communicate with others . 

The Semantic System of the Mentally 
Retarded 

Semantics or meaning in the lan- 
guage of retarded persons has impli- 
cations for how well they can under- 
stand the words, concepts, and ques- 
tions put to them. Studies which 
compare the vocabularies and lexical 
usage of retarded children and non- 
retarded children focus primarily . 
on such things as vocabulary size 
and diversity, degree of abstraction 
in word usage, word frequencies, and 
the relative usage of various parts 
of speech. In general, most studies 
indicate that retarded children use, 
significantly more concrete word 
forms and references than do normal; 
children of a comparable mental age 
(Yoder & Miller, 1972). For example, 
Sloan and Cutts (1947) determined . 
that the essential difference between 
"easy" and "difficult" items on th|B 
Stanford-Binet liitelligence Scale for 
retarded children was the degree of 
abstraction required to pass an item. 

Several early studies examined the re- 
' tarded person's ^ability to formulate 
abstract word definitions. Papinia 
(1954) demonstrated that as mental 
age increased j /the ability to define 
words in more/. abstract terms also 
improved for retarded persons. At 
any given menml age level retarded 
children could acceptably define as 
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many words as nonretarded children 
but their definitions were often 
simpler and more concrete. These 
findings suggest that retarded chil- 
dren do not differ qualitati.vely 
from normal chilcjren in terms of de- 
fining words, but merely develop the 
ability to provide more abstract de- 
finitions later* 

Badt (195B) has suggested that length 
of institutionalization may be a sig- 
nificant factor influencing the level 
of abstraction exhibited by retarded 
persons. Abstraction scores for a 
group of moderately and mildly re- 
tarded state school residents were 
negatively associated with the num- 
bery^of years each child had spent in 
the institution (r = -.61), while 
chronological age and mental age 
were only modestly related to level 
of abstraction = .34 and .24, re- 
spectively) . Badt concluded that in- 
stitutional living interferes with 
the retarded child's ability to form- 
ulate abstract relationships and to 
manipulate concepts. 

The inability of retarded children to 
express abstractions is also seen in 
their performance on tests which call 
for identifying similarities among 
different objects. Retarded children 
frequently insist on naming differ- 
ences rather than abstracting a com- 
mon feature from a group of items. 
For example, Griffith and Spitz (1958) 
found that retarded boys had trouble 
abstracting a common characteristic 
from a group of words similar on one 
dimension, even though they coi^ld 
adequately define the words individ- 
ually. . / 

Therefore, the evidence seems ;to in- 
dicate that retarded children /are 
less apt than normal children/ to use 
abstract concepts. In addition, 
there is some evidence to suggest 
that this ability to use language at 
a more abstract level becc:nes more 



impaired the lonaer a child remains 
institutionalized. This line of re- 
search, even though it deals with 
production rather than comprehension, 
suggests that retarded interviewees 
may have special difficulty under- 
standing questions which contain ab- 
stract words or concepts. 

The size and composition of an indi- 
vidual's vocabulary is another im- 
portant aspect of semantic develop- 
ment. Mein and O'Connor (1960) com- 
pared the oral vocabularies of in- 
stitutionalized retarded persons 
with those of nonretarded children 
of a comparable mental age. Re- 
tarded persons were found to average 
nearly one-third more words in their 
individual vocabularies than nonre- 
tarded children. Spreen (1965) re- 
viewed several studies confirming 
this observation that mentally re- 
tarded persons tend to have larger . 
vocabularies than nonretarded chil- 
dren of the same mental age. He 
attributes their larger vocabulary 
size to the fact that mentally re- 
tarded subjects are typically older 
and have had more exposure to lan- 
guage than their nonretarded coun- 
terparts. 

Looking at the kinds of words in the 
vocabularies of retarded and normal 
children, Mein and O'Connor found 
that institutionalized retarded 
persons share more words in common 
than do normal children of the Sd.ne 
mental age, possibly because insti- 
tutionalization causes them to have 
more experiences in common with 
their peers. Nonretarded children 
exhibited greater individuality in 
their vocabulary, despite the fact 
that their total vocabulary size was 
smaller^. It should be noted that 
in this study the size of a retard- 
ed individual's vocabulary did 
not seem to be affected by the 
length of institutionalization 
(r = .17) but was significantly 



2.13 

2S 



related to the retarded person's 
mental age (r=. 72) ; 

In a study comparing word fre'-^u^ncy 
and word usage of state sghool:: resi- 
dents and nonretarded children, Beir, 
Starkweather, and Lambsrt -'(1969) ob- 
served only minor differences in the 
types of words used by retarded and 
nonretarded children. For example, 
retarded children tended to use posi- 
tive words like "yes" and "O.K." more 
often (see the discussion of lacqui- 
escence that follows) . . Retai^ded child- 
ren also used a greater number of 
self -references (me, I, my, etc.) and 
fewer other-references, which is a 
pattern common in very young j children 
and indicative of social immaturity. 

Studying vocabulary, usage, in terms of 
common versus uncommon words, Lozar, 
Wepman & Hass (1972) found tjfiat the 
verbal output ^of retarded children 
was very much like that of noVretard- 
ed children. However, they^ also 
noted that there .w^r^ considei|able 
differences in the retarded cl|i^ld's 
usage of different parts of speech. 
They collected speech samples|from ^ 
institutionalized retarded^^chifdren^ 
and normal children matched on either 
mental age or chronological age by 
asking them to tell stories to TAT 
cards. The retarded children often 
spent more time describing th(5 cards 
which led to a Higher proportion of 
nouns, articles, and quantifiers in 
their speech. On' the. otKer han<3> the 
normal children were more likely to 
tell stories about the pictures, pro- 
ducing more verbs, prepositidns, and 
conjunctions. -j 

Previovi*3 research with nonretarded 
adults has suggested that thfe earlier 
a word ?s estimated to have been ac- 
quired by di ., rlividual, the faster 
the word is iv « : --^./rom memory 
(Carroll & Wliite, XJi3dL, 197'3b; Lach- 
man, 19737 Lachman, Schafferi, & Henn- 
rikus, 1974)^:1. -^^Therefore Winters and 



Brzoska (1975) have suggested that 
the sequence in which words origin- 
ally entered semantic memory can 
be estimated from their relative 
speech retrieval. Using this proce- 
dure, they found that retarded per- 
sons have smaller lexical stores 
than would be expected compared to 
nonretarded persons of a similar, 
mental age, but that they displayed 
a similar sequence of acquisition of 
lexical items An unexpected finding 
however, was that the, degree of lag . 
varied according to the class of 
the word, such that retarded per- 
sons were most deficient in words 
normally acquired earlier in the 
development sequence. This finding, 
though paradoxical, does support 
earlier findings that as mental age 
increases retarded persons show more 
diversity in their lexical usage 
(Bartel, Bryen, Keehn, 1973). 

In a follow-up to this study. Winters 
and Cundari (1979) compared the 
speed of retrieval of words from the 
lexicon of ihstitutionalized mental- 
ly retarded adolescents to the age- 
of-acquisition estimates derived 
from nonretarded adults. Words that 
were estimated to have been acquired 
the earliest by the normal group 
were retreived the f aste'st -by. the 
retarded group. The authors inter- 
preted these results as indicating 
that the verbal memory process of 
retarded and nonretarded persons 
are quite similar, despite the dif- 
ferences in lexical store size found 
by Winters and Brzoska (1975). 

Lay ton and Sharif i (1978) exarained 
the basic semantic structure under- 
lying sentences produced by Down's 
Syndrome children and nonretarded 
children. Down's Syndrome children 
used the same semantic features as 
did the nonretarded children, but 
with much less frequency or consis- 
tency. For example, both groups 
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produced-: the four basic semantic verb 
categories (state, process, action, 
and process-action) , but> the retarded 
group used the process (The road ends) 
and the process-action (John cut the 
paper) verb forms less frequently 
than the nonretarded children did. In 
contrast, there was a tendency for the 
retarded children to use more state 
(The road is rough) and action (John 
runs) verbs than the normal group did. 
This finding was interpreted as indi- 
cating that' Down's Syndrome children 
are more aware of or possibly restric- 
ted to the "here-and-now" aspects of 
their environment: "As children's cog- 
nitive abilities mature, their ability 
to perceive past and future events. im- 
prove; consequently, their expressions 
will include more semantic concepts 
depicting these changes" (p. 444). 
Overall, Layton and Sharifi concluded 
that there are more similarities than 
differences in the usage of verbs by 
Down's Syndrome and normal children, 
and the primary cause of delayed se- 
mantic development among Down's children 
is their immature level of cognitive 
developjiment. 

Recent research on how children use 
words to convey meaning suggests that 
retarded and nonretarded children are 
more alike than different in this di- 
mension. Young normal children have 
been shown to use single-word utter- 
ances for more complex purposes than 
the simple labeling of an object. For 
example, Greenfield and Smith (1976) 
suggest that young children call at- 
tention to a changing element or as- 
pect which is undergoing the great- 
est amount of change. Leonard, Cole, 
and Steckol (1979) reported that re- 
tarded preschool children label ob-r^ 
jects in much the same way as the non- 
retarded children in Greenfield and 
Smith's study to convey the greatest 
amount of information efficiently. 



Similarly, there is also some evidence 
to suggest that retarded children 
seem to impose meaning on verbal in- 
formation in much the same way as 
nonretarded children do. Nonretarded 
children frequently rely on context- ' 
ual information and their knowledge 
of real world relationships to inter- 
pret the meaning conveyed by a speak- 
er, Dewart (1979) designed a study tc 
determine whether mentally retarded 
children used strategies similar to 
those used by nonretarded children 
for interpreting the meaning of sen- 
tences. Three types of sentences were 
used that varied acceding to the 
kind of semantic cues they provided 
the child for use in interpretation: 
neutral , probable , or improbable. 
Sentences that were neutral with re- 
spect to semantic expectations would 
make sense if either noun served as 
the subject in the sentence. Probable 
sentences described likely events, 
while improbable sentences described 
events that were unlikely to occur in 
the real world (e.g.f The gate jumped 
the horse). If children were using 
semantic constraints to interpret these 
sentences, they would respond correct- 
ly more often to probable sentences 
than to neutral sentences, and least 
often to iiiiprobable sentences. Dewart 
concluded that retarded children are 
able to use contextual information 
and previous knowledge concerning the 
likelihood of a given event to guide 
their understanding of sentences in 
much the same way as nonretarded chil- 
dren of a comparable mental age do. 

In summary, mentally retarded persons 
appear to demonstrate less mature .Vex- 
ical development than normal children do. 
They show less abstraction, fewer other- 
references, and greater attention to 
semantic features that are anchored in 
the present. They typically define words 
in more simplistic and concrete terms 
because abstraction develops much later 
in retarded populations due to cognitive 
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limitations. Some studies indicate 
that retarded persons have a smaller 
lexical store than normal. Even v/hen 
they are shown to be sujg>erior in vo- 
cabulary size, they typically demon- r 
strate less variety and diversity in 
their lexicon. Despite these differ- 
ences, the sequence pf acquisition 
for lexical items has been shown to 
be the same as in normal development, 
only slower. The differences tend to 
diminish as mental age increases in 
the retarded population. 

In general, mentally retarded perscns 
appear to be able to understand and 
use semantic information in much the 
same way as nonre.tarded persons of 
the same mental age do. They have 
been shown to use the same types of 
semantic features in their speech as 
nonretarded children, even though 
they use them less frequently and 
with less consistency. The retarded 
also tend to use similar strategies 
for interpreting meaning and convey- 
ing information to others. Therefore, 
the semantic system of mentally re- 
tarded children develops normally al- 
though they tend to lag behind in 
their rate of acquisition. Retarded 
persons can generally be expected 
to perform much like younger normal 
children do, sometimes even more im- 
mature ly than their mental ages 
would cause one to predict. Conse- 
quently, in developing interview 
questions for mentally retarded con- 
sumers with low mental ages, one 
should, to the extent possible, use 
very simple sentence constructions 
which draw on concrete and commonly 
used words. 

LANGUAGE STRUCTURE OF THE MENTALLY 
RETARDED 

The structure of language used by 
mentally retarded individuals has re- 
ceived increasing attention in the 
literature over the past several 
years. This aspect of language acqui- 



sition includes the areas of morphol- 
ogy, or rules for forming words, 
and syntax, or rules for forming 
grammatically correct and meaningful 
sentences. Recently, there has been 
increasing emphasis on the mentally 
handicapped person' s ability to com- 
prehend langulage structures apart from 
his or her expressive abilities. In 
any attempt to obtain information 
from mentally retarded consumers 
about their needs, an understanding 
of their limitativ^ns in comprehending 
speech would be of great importance, 
since expressive requirements could 
be minimized by using various struc- 
tured interviewing strategies. 

Morphology 

Morphology is the study of word 
forms in a language. Berko's (1958) 
test of English morphology has be- 
come the most widely used method for 
studying the development of morphol- 
ogical rules in children, and has 
also been extended to investigating 
the pror'ess of morphological rule- 
learning in the retarded. For "exam- 
ple, the child is shown a picture of 
an imaginary object and is told "Here 
is a moo/c; now there is another one. 
There are- two of them. There are two 

The child must provide the 
correct plural form of the nonsense 
word. 

In reviewing' the studies which em- 
ployed Berko's technique with retard- 
ed children, Yoder and Miller (1972) 
and Cromer (1974) have concluded that 
retarded subjects matched on mental 
age learn the rules of morphology in 
a similar manner to normal children, 
but at a slower pace. However, some 
researchers contend that the retarded 
also differ from normals in their 
ability to generalize these irules to 
new, unfamiliar stimuli (Leiber & 
Spitz, 1976: Lovell & Bradbury, 1967) 

A direct test of the rule-learning 
deficit hypothesis was made by Bartel 
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(1970) with a group of mildly and 
moderately retarded children. She 
predicted that if such a deficit ex- 
isted in the retarded population, 
moderately retarded children would 
improve on specific words when train- 
ed to provide the correct inflections 
for them, but the improvement would 
not generalize to other words. Con- 
trary to her hypothesis, the two IQ 
groups in the study generalized to 
untaught items equally well. 

In light of evidence provided by 
Dever (1972) on the predictive val- 
idity of Berko's procedure with the 
mentally retarded, much' of the pre- 
ceding information must be interpre- 
ted with caution. Consistent with 
previous findings, he and Gardner 
(1970) found that retarded children 
did not perform as well as normals 
on the Berko test even when matched 
on mental age. However, they observ- 
ed that during spontaji^eous conver- 
sation some of their i-etarded subjects 
demonstrated appropriate use of many 
morphemes, even though they had per- 
formed poorly on the Berko task. 
Dever (1972), therefore ^ used a re- 
vised version of Berko 's test to 
determine whether performance on ei- 
ther part of Berko' s test could pre- 
dict the free speech errors of re- 
tarded subjects. Since it could not, 
Dever concluded that the Berko test 
is not a valid diagnostic test of 
morphology usage in the mentally re- 
tarded. 

However, it 4s important to note 
that retarded and normal groups found 
the same items to be relatively dif- 
ficult in. Dever and Gardner • s, study . 
plurals and possessives were easiest, 
followed by progessive verb forms. 
Irregular verb forms were not hand- 
■ led well by any of the children be- 
fore mental age of 10. Therefore, 
despite the lack of predictive valid- 
ity of this test with respect to 
. morphological usage in spontaneous 



speech, it appears that retarded 
children develop understandings of 
morphology in the same sequence nor- 
mal children do. Overall then, while 
thei mentally retarded may sometimes 
demonstrate less flexibility in gen- 
eralizing morphological rules to new 
situations than normal children, 
there seems to be no strong support 
for the notion of a rule-learning 
deficit in retarded children. In 
addition, Graham and Graham (1971) 
reported that even severely retard- 
ed persons produce a considerable 
amount of speech which is formulated 
on the basis of appropriate grammat- 
ical rules and operate at a level 
comparable to that of normal chil- 
dren, of the same mental age. 

Syntax 

In studies of syntax development 
in children one method that is fre- 
quently used is the cloze procedure, 
which requires the child to supply a 
word missing from a sentence. Perfor- 
mance on this task is determined by 
the child's ability to correctly 
judge the grammatical class of the 
word and then to select an approp- 
riate word from that class to com- 
plete the sentence. Research has 
demonstrated that retarded children 
are more likely to use sequential 
strategies rather than paradigmatic 
strategies in making word associa- 
tions (Semmel, Barritt, Bennet, & 
Perfecti, 1968) . That is,. like young 
normal children, they associate 
words on the basis of contiguity (e. 
g., red-blue). For this reason, Sem- 
mel, Barritt, and Bennet (1970) pre- 
dicted that retarded subjects would 
find the cloze procedure more dif- 
ficult than the nonretarded children. 
They also predicted that when a word 
was omitted from the end of a sen- 
tence, retarded children would suq- 
ceed more often since more semantic 
clues would be provided to guide the 
selection of the missing word. 
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Retarded children ages 10 to 14 years 
with an IQ range of 60 to 80 were 
matched with two groups of nonretard- 
ed children: one on chronological age 
and one on mental age. In general, 
the retarded children did more poor- 
ly than either nonretarded group in 
the use of specif ic "grammatical form 
classes. As predicted, the retarded 
children also showed greater improve- 
ment in performance when supplying 
words omitted from the end of a sen- 
tence than normal children, suggesting 
that the reta^rded children were more 
dependent on sequential strategies 
for selecting words. However, in re- 
viewing this study, Cromer (1974) ob- 
served that there was sufficient 
similarity in the performance of the 
retarded children and the normal 
children matched on mental aye to 
suggest that the retarded group was 
not responding with deviant strat- 
egies. For example, both the retard- 
ed and nonretarded children found 
nouns the easiest to replace. 

Goodstein (1970) replicated Semmel^ 
et al. (1970) study with the addi- 
tion of a recognition version of the 
cloze procedure. These results in- 
dicated only quantitative differences 
between retarded and normal children. 
Both studies,, then, suggest that 
sentence complexity affects sentence 
comprehension among the retarded in 
much the same way as it does among 
normal children. Retarded children 
did not make different kinds of errors, 
just mpre of them. Consistent with^ 
the findings of Seimnel et al. (1970) 
Goodstein found that the order of sen- 
tence difficulty was the same for both 
the retarded and nonretarded children. 

Several researchers have attempted to 
determine if the kinds of sentence 
con'Structions that normal children 
and adults have difficulty understand- 
ing also cause problems for the re- 
tarded. Semmel and Dolley (1971) 
studied a sample of Down's Syndrome 
children , ranging in age from 6 to 14 



years with an IQ range of 22 to 62. 
They presented their subjects with 
both comprehension and imitation 
tasks. using the four sentence types, 
and expected to replicate the order 
of sentence difficulty reported by 
Slobin (1966) for normal children 
and adults. In the comprehension 
task, subjects were presented with a 
pair of pictures, each representing 
one aspect of a reversible situa- 
tion (e.'j., A clown hitting a ball 
and a ball hitting a clpwn) . As pre- 
dicted, Down's Syndrome children 
performed best on the simple de- 
clarative sentences (slightly above 
chance) . Performance on passive and 
negative passive sentences was only 
at a chance level of responding. 
Surprisingly, Down's Syndrome chil- 
dren performed below chance on nega- 
tive sentences.- The authors suggest- 
ed that these children were respond- 
ing to negative- sentences as if they 
were simple affirmative sentences, 
and proposed that Down • s . Syndrome 
children may be unable to process 
negative sentences, or may simply 
ignore the negative marker in these 
sentences. As a result, Slobin 's 
findings were not replicated in this 
group of retarded children. 

In contrast, Lamberts and Weener 
(1976) reported that over half of 
the retarded subjects in a language- 
training project successfully repeated 
sentences which contained a negative 
marker. Since previous research has 
demonstrated that retarded children 
correctly imitate only those sen- 
tence structures which already exist 
in their own spontaneous speech pat- 
terns or which they are able to un- 
derstand (Lackner , 1976 ; Lenneberg , 
1967) , the hypothesis that retarded 
children cannot process negative 
sentences seems less plausible. Moreover 
in the Lamberts and Weener study, compre 
hension of negative and affirmative sen- 
tences was related to sentence revers- 
ibility (that is, to whether subject and 
object can be interchanged, as in "the 



2.18 



31 



cat chased the dog," or whether they 
are irreversible as in "the cat climb- 
ed the tree"). Nonreversible nega- 
tive sentences were actually easier 
to understand than reversible affir- 
mative sentences for retarded per- 
sons with a mental age of five or 
above . 

Semmel and Dolley (1971) did not 
find sentence comprehension to be 
related to level of intelligence. 
However, later research has reported 
that the ability to comprehend sen- 
tences in retarded children does vary 
with mental age (Bartel , Bryen, & 
Keehan, 1973; Lamberts & Weener, 1976; 
Walker, Roodin, & Lamb, 1957). Lamb- 
erts and Weener note that, on the 
average, the subjects in Semmel and 
Dolley •s sample were much younger 
than their sample of retarded chil- 
dren. Therefore, comprehension of more 
complex sentence constructions may 
not have developed in Semmel and ^ 
Dolley 's Down's Syndrome children 
because a requisite levol of cogni- 
tive or linguistic functioning had 
not been attained. Semmel and Dolley 
(1971) did find level of intelli- 
gence among Down's children to be 
strongly associated with performance 
on a sentence imitation task. 

Mittler (1970) proposed that sen- 
tence comprehension depends more on 
structural complexity than on sent- 
ence length. He reported that the 
rank order of sentence difficulty 
was similar in a group of. retarded 
children and a matched sample of 
nonretarded children. An analysis 
of errors suggested that comprehen- 
sion was most affected by the struc- 
tural complexity of the sentence in 
both groups of children. Berry (1972) 
examined the effects of structural 
complexity on sentence comprehension 
when sentence length was controlled. 
He compared the comprehension ability 
for retarded and nonretarded children 
for simple declaratives (e.g.. The 
boy is fishing here.) and more com- 



plex possessive sentence construc- 
:tions (e.g.. The boy's fish is here.) 
Possessive sentences Wv^re more 
difficult for both groups of chil- 
dren, ^ , 
A number of other studies have shown 
that retarded children are generally 
able to process simple declarative 
sentences with the same accuracy as 
nonretarded children of a comparable 
mental age. However, as transforma- 
tional complexity increases, the 
retarded child lags further behind 
his or her mental age counterpart in 
sentence comprehension. For example, 
Dewart (1979) reported that retarded 
children abctve mental age three com- 
prehend simple active sentences' above 
chance, while passives are inter- 
preted at a level below chance. She 
indicated that their, performance 
was very similar to that of a group 
of nonretarded children whose av- 
erage mental age was lower, which ^ 
supports the idea of delayed acqui- 
sition of passive constructions in 
the retarded. Retarded children 
with a lower mental age, however, 
performed at a chance level on both 
active and passive sentences- 
Therefore, Dewart concluded that, 
prior to mental age three, retarded 
children are quite limited in their 
ability to understand and use 
syntax . 

Despite the fact that the acquisi- 
tion of more complex ^sentence struc- 
tures is delayed in the language 
development of the mentally retarded, 
such learning does occur. The order 
' of acquisition of complex sentence 
constructions parallels that ob- 
served in the developmental sequence 
of normal children ' (Berry & Foxen, 
1975; Gallagher, 1969) . For example, 
Graham and Graham (1971) collected 
language samples of nine institution- 
alized retarded males, ages 10 to 18 
years, with mental ages ranging from ' 
3-6 to 10-0. Retarded males with 
lower mental ages produced most of 
their sentences through the use of. 
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base strings or kernel sentences 
with no transformational rules ap- 
plied. Mental age was highly corre- 
lated with use of transformations to 
form more complex sentences. Thus 
(Graham and Graham concluded that 
language facility does depend pri- 
marily on intellectual level in 
retarded persons and that retarded 
children develop the rules of lan- 
guage in much the same manner as 
nonretarded children only at a slow- 
er rate. 

In one of the most linguistically 
sophisticated studies to date on the 
language development of retarded 
children^ Lackner (i976) has provid- 
ed additional support for this view- 
point. He collected language samples 
from five retarded children who 
represented five ascending levels of 
intellectual functioning and com- 
pared their linguistic capabilities 
with those of nonretarded children. 
The order of appearance of sentences 
in the language patterns of the re- 
tarded children was consistent across 
the five mental age levels and fol- 
lowed a regular order of increasing 
structural complexity : declaratives, 
negatives, questions, negative ques- 
tions, passives, negative passives, 
and negative passive questions. No 
sentence type of a given order of 
complexity was found in the retard- 
ed child's grammar unless all lower 
order constructions were also part 
of his or her language repertoire. 
More significant was the finding 
that as mental age increased the 
number of sentence transformation 
rolds understood and used by each 
child also increased, indicating that 
language development as measured by 
transformational complexity does 
improve with higher intellectual 
functioning. This supports the pre- 
vious findings of Graham and Graham 
(1971) . Lackner also observed that 
none of the sentences of the retard- 



ed children were incompatible with 
normal adult usage, supporting the • 
idea that retarded persons do not use 
deviant language structures. Retarded 
and normal children exhibited simi- 
lar developmental trends, but the 
language development of the most 
severely retarded children appeared 
to be arrested at a level lower than 
that of their mental age-matched 
counterparts . 



In contrast, some studies with men- 
tally retarded persons have reported 
different and conflicting informa- 
tion. For example. Guess (1969) and 
Guess and Baer (1973) concluded that lan- 
guage reception and production in 
mentally retarded children may be 
j.ndependent processes. They fo\and 
that training retarded subjects in 
one modality did not produce general- 
ization effects in the other language 
modality . Similarly, Miller , Cuvo, 
and Borakove (1977) found that teach- 
ing reception skills to the retard- 
ed did not improve expression; how- 
ever, training in production did im- 
prove receptive skills. This research 
suggests that a qualitative differ- 
ence may exist between retarded and 
nonretarded populations with respect 
to the direction of generalization be- 
tween comprehension and production. 



There has been much debate in the 
literature over the relationship be- 
tween comprehension and production 
in the language acquisition process. 
Current thinking assumes that compre- 
hension precedes expression or pro- 
duction in the developmental sequence. 
Research on language development of 
nonretarded children, supports this 
notion (Fraser, Bellugi, & Brown, 
1963; McCarthy, 1954). Other recent 
studies with nonretarded children 
have also demonstrated that prior 
training in comprehension skills fac- 
ilitates acquisition of production 
abilities (Vasta & Teitelbaum, 1976? 
Whitehurst, 1977). 
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Lackner (1976), however, feels that 
production of grammar lags behind 
comprehension in the mentally re- 
tarded. He observed that both re- 
tarded and nonretarded children in 
his study frequently modified com- 
plex sentences on an imitation task 
in such a way that the meaning of 
the sentence was retained but sen- 
tence complexity was reduced. For 
example, a child would sometimes re- 
peat the passive sentence construc- 
tion "Mary was hit by the ball" 
as "The ball hit Mary." Lackner 
felt that this indicated that the 
child's productive skills for gram- 
mar were not as well developed as 
his ability to understand syntax. 

Similarly, Walker, Roodin and Lamb 
(1975) reported that expressive 
language consistently lagged behind 
comprehension in a sample of re- 
tarded children with a mental age of 
3 to 7, 5. In a group of nonretarded 
children with a comparable mental 
age range, this difference occurred 
for only the youngest children, 
ages 3 to 3.5. As mental age in- 
creased in the retarded group, the 
magnitude of the difference was 
found to decrease so that by mental 
age seven expressive and receptive 
abilities were nearly equal. The 
authors concluded that expressive 
tasks may place a greater demand on 
short-term memory than comprehen- 
sion tasks do. In their -research, 
"maximal estimates of language 
ability in retarded subjects were 
obtained in tests that measure 
comprehension without requiring 
verbal production by the child" 
(p. 551). Consequently, one might 
expect that maximal information 
could be obtained from retarded per- 
sons using interview questions 
which make minimal demands for 
verbal production. 

In summary, then, the research evi- 
dence suggests, that the language 



structures of mentally' retarded 
persons are quite similar to 
those of nonretarded children at a 
similar mental age. In addition, 
there is no strong evidence tp 
suggest that the mentally retarded 
use language structures or strat- 
egies that are incompatible with 
those of normal children, even 
though language acquisition fre- 
quently occurs at a much slower 
rate. Language delays are likely 
to be greatest for retarded persons 
below mental age seven, since 
prior to this level of functioning 
they do not seem to possess the 
cognitive capabilities necessary 
for comprehending the more complex 
aspects of grammar. In addition, 
retarded persons below mental age 
three would be expected to have 
very little xanderstanding of cor- 
rect grammatical usage, since 
retarded persons have been shown 
to consistently perform best on 
language tasks involving simple 
sentence structure, this would 
suggest that unnecessary complexity 
should be avoided when questions 
are asked of the retarded. 

Question Development in Children 
and Its Implications for the 
Retarded 

No research that we are aware 
of has directly examined the devel- 
opment of question-answering com- . 
petence in retarded individuals. 
However, research on normal chil-:. 
dren's comprehension and use of 
questions has been reported within 
the last 15 years, and it is highly 
relevant to our concern with^ how 
to phrase question's. The evidence 
suggests that there is a basic 
sequence of question development 
in normal children., although indi- 
vidual studies have shown some 
variation in the' order of mastery. 
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It seems' reasonable to expect that 
a similar pattern of question de- 
velopment occurs in the mentally 
retarded f since their language pro- 
"^^esses have not been shown to be 
qualitatively different from those 
of nonretarded children. 

By age four, children are typically 
iable to respond appropriately to 
most question forms. Bellugi (1965) 
observed that before age two cnildren 
do not seem to understand many of 
the wh-questions. Comprehension 
for these question formats begins 
to emerge somewhere aroimd 18 to 
28 months of age, with yes-no, ' 
what- and where-questions generally 
being the first to be understood. 
Why- and who-questions were among 
the next questions to be acquired 
(aroxind the age of 32 months) . 
These observations correspond very 
closely wi^.:h '^arly study by 
Smith (1933), who found that what, 
where, and yes-^no question types 
were the most frequently produced 
by children ages 1-G to 6-0. 

Ervin-Tripp (1970) conducted the 
first ;aajor study cf how children 
coiar ;i:.end questions. She observed 
the raspon'ses tc- questions of chil- 
dren ages 2.3 to 3.1 over £. one-year 
periodc Ini'cially, all children 
could answer what, where, and yes- 
no questions appropriately, which 
agrees with Bellugi "s observations. 
' Hhe relative order of acquisition 
for other wh-questions , according 
to Ervin-Tripp was: why and who- 
subject; how and where-from; when 
and who- subject questions. How- 
ever, since this order was not 
stable across all children in the 
study, she presents this as a ten- 
tative developmental sequence for 
question acquisition in normal 
children. 

The level of the child's cognitive 
development appears to influence 



what questions he or she can answer. 
For example, aroiind the age of three, 
most children begin to produce ad- . 
equate causal responses, to simple 
why questions that are manageable 
at their level of cognitive func- 
tioning (Bellugi, 1965; Ervin- 
Tripp, 1970). By comparison, the 
child's \inderstanding of time 
concepts does not develop fully 
\intil after age six (Piaget, 1927), 
which would explain why when-queStions 
are among the mofet difficult for 
young children to \inder stand. 

Recently, Tyack and Ingram (1977) 
attempted to replicate the findings 
of Ervin-Tripp in a study of chil- 
dren's question comprehension and 
production. The children in the 
production experiment used yes-no, 
what, and where questions most 
frequently, with the exception of 
3-6 to 3-11 group who used why and 
how questions more frequently tham 
where questions. The authors 
suggested that what and where ques- 
tions are used frequently by yo\ing 
children because they are closely 
tied to the child's immediate ex- 
perience. An alternate explanation 
has been suggested by Hooper (1971) , 
who proposes that yes -no and label- 
ing questivOns like "What is that?" 
are much easier for children to 
answer thsm open-ended smd explana- 
tory questions such as how and why 
because they make fewer demands 
for grammatical processing and verbal 
elaboration • Possibly both this 
linguistic explanation and the cog- 
nitive development viewpoint accoiint 
for the earlier use and comprehen- 
sion of yes-no and what questions 
in children. 

While frequency of use does nq>tr . 
necessarily reveal the order of ^ 
question acquisition in children, 
the usage figures reported by Tyack 
and Ingram (1977) do agree with the 
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comprehension data of Bellugi (1965) 
and Ervin-Tripp (1970). However, 
the acquisition sequence for ques- 
tion comprehension which emerged 
from Tyack and Ingram's sample of 
3.0 to 5.5 year olds was somewhat 
different from that reported in 
earlier s'^udies. The order of ques- 
tion type was as follows: where- 
intransitive (94%); why-intransitive 
(83%) ; why-trans.ltive (81%) ; 
who-subject (80%) ; where-transitive 
(67%); './hat-object (57%) , who-object 
(56%) ; when-intransitive (52%) ; 
and when-transitive (48%) ; hoV- 
transitive (38%) ; how-intransitive 
(35%), and what-subject (35%). 
According to these findings, intran- 
sitive verb forms are better under- 
stood than transitive forms, except 
in how-questions. As the authors 
note, when-questions were easier to 
answer than how-questions for these 
subjects, where this was not the 
case in Ervin-Tripp ' s study. A 
more obvious discrepancy was that 
what-questions were more difficult 
to answer in this study. In fact, 
what-subject questions were actually 
the, most difficult to understand 
for these children. The authors 
suggest that young children may . 
automatically associate what 
questions with the object. More 
generally, Tyack and Ingram con- 
cluded that transitivity and the 
semantic features of a verb are as 
important in determining the child's 
response to a question as is the 
question format used. 

The research evidence on the acqui- 
sition and comprehension of questions 
in noirmal children "seems to suggest 
that the mentally retarded, especially 
those who are severely retarded, 
would find what, where, and yes-no 
questions the easiest to comprehend 

. and answer. They would probably 
find who, how, and why questions 

' more difficult to comprehend and 



even more difficult to answer be- 
cause of the greater demands for 
verbal expression made by thes.e 
question formats . When-questions 
would probably be the most difficult 
to answer because of the higher 
level of cognitive functioning nec- 
essary to understand time con- 
cepts. However, as Tyack and Ingram 
have indicated, the transitivity 
and semantic features of the verb 
used could also affect this order 
of difficulty significantly. 
Since level of cognitive develop- 
ip.ent appears to be an important 
factor in the normal child's 
ability to respond to questions, 
one would expect retarded children 
to acquire these question forms 
at a slower rate, especially if they 
are severely retarded. 

Response Effects in Survey Research 

In any research, including re- 
search using interview techniques , 
validity of measurement is a 
critical issue. Simply stated, 
one needs to know what an inter- 
view measures; that is, whether re- 
sponses to an int'ferview accurately 
portray the actual behaviors or 
attitudes of the individual inter- 
viewed. Since part of the purpose 
of our research was assessment of 
the validity of information ob- 
tained from mentally retarded re- 
spondents , we sought guidance from 
the literature on measurement and 
measurerrtent error in survey research. 

Traditionally it has been assvimed 
that responses to an interview are 
composed of two factors. The first 
is the respondent's "true answer," 
or the actual information sought by 
the interviewer. The second is 
response error, or variability 
in the respondent's answer due to 
all sources other than the "true 
answer." Response error, then, is 
the difference between the "true 
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answer" and the answer actually 
given in the inteirview. Response 
error arises from many sources; 
a response effect is the response 
error due to one specific factor. 
If response effects tend to cancel 
one another out so that mean re- 
sponse error is zero, straight- 
forward iiiterpretation of the 
interview data for a group is pos- 
sible. If, however, some response 
effects are large enough that the 
obtained answer consistently" 
varies in a given direction from 
the "true answer," interpretation 
of the intearview data is much more 
difficult. Data from our research 
suggest that such powerful response 
effects may be at work in inter- 
views' with -mentally retarded indi- 
viduals , tending to reduce the 
validity of answers. The litera- 
ture review which follows is an ef- 
fort to assess the relative impor- 
tance of response effects due to 
various factors , with particular 
emphasis 9n those variables ex- 
pected to have a special importance 
in intearviews with mentally re- 
tarded persons. We will be con- 
cerned with learning whatever les- 
sons may be learned, from siirvey 
research with the .general popula- 
tion that might aid in interview- 
ing mentally retarded respondents. 

Fortunately, Sudman and. Bradburn 
(1974) have provided a review and 
analysis of research .of response 
effects which is a landmark in the 
field. These authors used a coding 
strategy to combine results of an 
enormous number of inteirview 
studies and estimate the relative 
importance of various response 
effects. The present review is, 
in many respects , a summary of the 
major conclusions of the Sudman and 
• Bradburn work, with emphasis and 
elaboration of those biases that 
seem potentially most relevant to , 



interviewing mentally retarded 
persons. In particular, the model 
of the interview situation and 
the conceptual framework for 
evaluating different types of 
, response effects are adopted 
directly from Sudman and Bradburn. 
This model views the inteirview 
as composed of three elements: 
the task of giving and receiving 
information, the role of the inter- 
viewer, and the role of respondents. 
Thus, the model regards response 
effects as a function of either 
task variables , interviewer vari- 
ables, or respondent variables. 

Task* variables are the form and 
conditions of the interview itself. 
The specific characteristics of 
the task may be expected to exert 
a tremendous influence on the 
nature of the information obtained. 
Question wording is an important, 
task variable. Bias may be intro- 
duced directly by^ the wording of 
a question, or may result indi- 
rectly from the length or diffi- 
culty of questions. One might also 
expect the structure of questions 
(e.g., closed-ended vs. open-ended) 
to exert an effect on responding. 
Questions which offer the possibility 
of an agreeing response can give 
rise to powerful acquiescence 
effects, a tendency to agree or 
say "yes" regardless of question 
content. Another task variable is 
the degreie to which an interview 
raises issues of self-presentation 
or, more specifically, creates 
in respondents the desire to 
present themselves in a favorable 
light. Whether an interview is 
directed towards behavioral or 
attitudinal information is another 
task variable of interest.. Still 
another may be described as the in- 
formational demands of the interview. 
Questions may place greater or 
lesser demands on the 'respondent ' s 
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memory, and may pertain to topics 
of greater or lesser salience to the 
respondents. Finally, the method of 
administration (for example, face- 
to-face vs. self-administered inter- 
views) is potentially important. 

Interviewer and respondent variables 
generally refer to characteristics 
of the individuals involved, such as 
their age, race, sex, and level of 
education. The magnitude of response 
effects due to interviewer character- 
istics is usually dependent on 
respondent characteristics and the 
topic of the interview. Similarly, 
the importance of respondent var- 
iables typically depends on inter-^ 
actions with task and interviewer 
variables. In fact, although for * 
the sake of clarity Sudman and Brad- 
burn describe task, interviewer, and 
respondent variables as separate 
sources of response error, their 
data show clearly that most impor- 
tant response error is a function of 
combinations of these variables. For 
example, while question wo}.'-*ing and 
the education of the respondent 
can each give rise to response ef- 
fects, the response effect due to 
the interaction of these two var- 
iables is larger. 

TASK VARIABLES 

The tren,d in survey research 
has long been to neglect task vari- 
ables in favor of investigations 
of interviewer and respondent 
characteristics. The most general 
conclusion of the Sudman and 
Bradburn review is that this em- 
phasis is unwarranted; in fact, the 
most general and important response 
effects arise 'from task variables. 
The specific task variables found to 
be of importance are evaluated below. 
Much of our own research effort fo- 
cused on the effects of task vari- 
ables such as question structure and 
wording . 



Question Wording 

The specific wording of a ques- 
tion might be expected to give rise 
to response effects in two general 
ways. First, bias inherent in t^he 
wording, as in the use of emotion- 
ally laden or leading words, may 
alter ^the respondent's answer. Sec- 
ond, difficult or ambiguous ques- 
tions may enhance response effects 
due to other variables. Essential- 
ly, if the respondent cannot inter- 
pret the question, he or she cannot 
give the "true answer." 

Good evidence exists that responses 
can be systematically biat^ed by 
question wording. For instance, 
Rugg (1941) asked the questions: 
"Do you think the United States 
should allow public speeches 
against democracy?" and "Do you 
think the United States should for- 
bid public speeches against democ- 
racy?" In response to the first 
question, 62% of respondents be- 
lieved such speeches should not be"' 
allowed, while in answering the 
second, only 46% were willing to 
forbid speeches. Clearly, the word- 
ing of these questions had a strong 
influence on the response effects 
due to question wording. 

Schuman and Presser (1977) repli- 
cated and extended these results. 
Using Rugg's original questions, 
they found response effects due to 
questions wording to be of similar 
magnitude and direction. Addition- 
ally Schuman and presser challenged^ 
the assumption, implicit in much re- 
sponse error literature, that ques- 
' tion wording and respondent charac- 
teristics do not interact. If this 
assumption were correct, then re- 
sponse effects due to question word- 
ing wouldj not affect comparisons 
of subgroups of respondents. How- 
ever, Schuman and Presser showed 
that less educated respondents were 
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more influenced by question wording 
than were more educated respondents. 
Among respondents with 13 or more 
years of education, the "allow" and 
"forbid" forms of the question 
yielded estimates o:^ the proportion 
of respondents opposing speeches 
which differed by only 5.9%. This 
difference was 25.6% for respondents 
with 0-11 years of education. In 
short, question wording can l*ave 
powerful response effects, and these 
response effects are greater for less 
educated respondents. These results 
may have important implicatiori-s for 
the interviews with mentally retarded 
individuals, who are at the low end 
of the educational continuum. 

The length and difficulty of ques- 
tions also may affect responding. 
Length of question, simply defined 
as the number of words, has generally 
had no overall effect on responding 
in face-to-face administration. Of 
•more interest for our research is the 
interaction of question length with 
respondent variables. Respcise ef- 
fects for respondents with a high 
school education or less are largest 
for questions more than 18 words 
long. Elementary students are par- 
ticularly influenced by long ques- 
tions. This finding may have impor- 
tant . implications for interviewing 
mentally retarded persons; if long 
questions cause large response ef- 
fects in younger and leSs educated 
respondents, it m^y be particularly 
important that questions posed of 
mentally retarded individuals be 
short and clear. 

The number of letters per word has 
been used as an indicator of question 
difficulty. Difficulty of questions 
has no. general effect for attitudinal 
itemj^; although response effects for 
threatening behayorial i*tems increase 
for questions using longer words. 



Question Structure 

The issue of question structure 
is typically posed as a choice be- 
tween closed-ended questions, which 
ask the respondent to choose from 
a fixed set of response alternatives, 
and open-ended questions. Although 
Sudman and B-adburn found no consist- 
ent effect due to question struc- 
ture, they did find that response 
effects due to issues of self- 
presentation are larger for closed- 
ended than for open-ended questions. 
That is, for threatening topics or 
topics for which there is a socially 
desirable answer ^ closed-ended ques- 
tions increase response effects. 

in addition, direct comparisons of 
closed and open-ended surveys have 
demonstrated that different 
forms of a survey can yield very 
different results. Jenkins (1935) 
reported a study which involved con- 
struction of an exhaustive checklist 
by using all responses to a previ- 
ously administered open-ended survey. 
This checklist (including an "all 
others" items) yielded response 
patterns closely comparable to the 
open-ended survey. However, when 
some of the popular items" were 
deleted from the checklist, response 
patterns to the incomplete check- 
list form differed radically from 
responses to the open-ended form. 
It is. at least clear that the make- 
up of a checklist can have important 
influences on responding, ancl l^hat 
if items often mentioned in response 
to open-ended questions are omitted, 
under-reporting may result. 

Belson and Duncan (1962) compared 
checklist and open-ended surveyr 
with regard to TV programs Watched 
and periodicals read by respondents 
during a specified period preceding 
the interview. They reported far 
higher claims of activity in response 
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t;o the checklist form than to the 
open-ended survey, with the size, of 
differences between question forms 
varying across items. Also, Belson 
and Duncan included some checklist 
items naming TV programs not actu- 
ally televised during the period 
of inquiry. A small fraction of 
respondents claimed to have watched 
these programs, demonstrating that 
5» in at least some instances closed- 
ended surveys can cause overreport- 
ing. 

Schuman and Presser (1977) also 
reported large differences on some 
items between endorsement rates on 
open and closed forms. Response 
effects due to question structure 
were found to be related to the 
education of the respondent. For 
example, the poorly educated were 
more likely than other respondents 
to endorse a job security item on 
the closed-.ended form, but 
less likely to list the same item 
on the open-ended form. Response 
effects were also larger for the 
less educated in a comparison of 
agree-disagree vs. forced choice 
questions. 

In summary, although Sudman and 
Bradburn observed no general re- 
sponse effect due to question struc- 
ture, direct comparisons reveal sub- 
stantial response differences be- 
tween, closed and open-ended ques- 
tions. Closed-ended (checklist) 
questions tend to provide a higher 
yield of clai:ns , which in some 
instances may constitute overreport- 
ing. Finally, response effects due 
to (question structure may be exag- 
gerated in less educated respond- 
ents, another hint that special 
problems may arise in interviewing 
retarded persons. 

Acquiescence 

Questions which allow for the 
possibility of an agreement response 



"yes," "true," "agree," and the like, 
may give rise to acquiescence, or a 
general tendency to agree independ- 
ently of item content. Since 
acquiescence was examined in the 
present study, we will consider it - 
in some detail. 

unlike most response effects dis- 
cussed thus far, acquiescence is a 
topic of interest primarily within 
personality research. The large body 
of literature on acquiescence (see 
Cronbach, 1946, for his influential 
introduction of the issue) emphasizes 
the effects of acquiescence on the 
interpretation of personality inven- 
tories like the California F scale 
and the MMPI. For example, items on 
the F scale are all keyed true; it 
has been suggested (see, for example, 
Bass, 1955) that the F scale identi- 
fied particularly acquiescent indi- 
viduals, rather than authoritarian 
individuals as cirgued by the creators 
of the scale. However, this body of 
literature arrives at no consensual 
conclusion regarding the importance 
of acquiescence in personality inven- 
torieis (see Rorer, 1965, for a criti- 
cal review ,of the literature) . 

No consensus has been reached on this 
issue Icirgely because it is a meth- 
odologically diffiv-^ult one. In order 
to evaluate the importance of acqui- 
escence in any instrument, one must 
somehow separate question form and 
question content — ordinarily highly 
confounded factors. This separation 
is pcirticuiarly difficult with per- 
sonality inventories, where the "true 
answer to an item is now known and 
where items are usually attitudinal 
and rather complex. It is therefore 
not particularly suprising that the 
role of acquiescence in these inven- 
tories is unclear. 

Our research is directly concerned 
with acquiescence as: a f actor which 
might invalidate responses to qvies- 
tions much less ambiguous than those 
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typically found on personality in- 
ventories. The questions we are 
concerned with tend to be behavioral 
(Do you date?) or relatively 
simple attitudinal (Are you usually 
happy?) questions. As we will dem- 
onstrate later f acquiescence may 
be a highly significant biasing 
factor in interviewing mentally 
retarded persons. The research de- 
scribed below is organized around 
four approaches to assessing acqui- 
escence • 

Content-free Measures 

One strategy for demonstrat- 
ing the effect of acquiescence is 
to employ "interviews" which are 
essentially content-free. In this 
situation, any non-random response 
patterns observed must be responses 
to the form rather than the content 
of the question. For example, 
Berg and Rapaport (1954) gave s\ib- 
jects a set of response alterna- 
tives without any questions. They 
were instructed to try to guess 
the correct answers. Berg and 
Rapaport observed clearly non- 
random patterns of responding to 
this essentially content-free 
measure. Of particular interest 
is the fact that response alterna- 
tives "yes/" "satisfied," "true," 
and "agree" were significantly 
favored by respondents regardless of 
the order in which alternatives were 
presented. 

In a similar study, Gerjouy and. 
Winters (1966) tested response 
preferences in 60 institutionalized 
educably retarded adults. They 
presented the subjects with pairs 
of identical geometrical figures 
and asked if the figure to which 
they pointed was the larger (or 
smaller). Of all responses, 59.1% 
were "yes," significantly different 
from chance at the .001 level. 



Clearly, a pi'eferencQ for agree- 
ment responses exists in both nor- 
mal and retarded individuals in 
an extremely unstructured situ- 
ation. It is conceivable that 
questionnaires may essentially 
be unstructured or "content-free" 
for mentally retarded respondents 
if the demands of the questions 
exceed intellectual abilities; in 
.such a case, responding would 
probably tend to resemble the 
acquiescence observed in the above 
studies . 

Tests with Known Answers 

A second strategy, one employed 
in the present study, is to use 
items to which the correct answers 
are known. If in this situation 
the pattern of responding differs 
systematically from the pattern of 
correct answers , a response set 
may be inferred. Cronbach -(1942) 
used this approach with true-false 
tests and observed that students 
in general are inclined to guess 
"true" when in doubt • Further, - 
Cronbach found that each student's 
tendency to acqviiesce was somewhat 
consistent across tests; a given 
student's tendency to answer "true" 
correlated from .36 to .61 across 
tests . Interestingly , if students 
guess "true" when in doubt, then 
false items have greater validity 
than true items as measures of the 
students' knowledge. In fact, 
Cronbach showed that in some cases 
the false items have greater 
validity than the true and false 
items combined. This finding may 
be important in survey research 
with particularly acquiescent 
populations; from Such persons, 
a "no" response would, on the 
average , be more meaningful than 
a "yes" response. In short, in- 
dividuals have a somewhat stable 
tendency to answer "true" when in* 
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doubt, and this fact tends to 
make false items more valid than 
true items • 

The Item-reversal Approach 

Unfortunately, neither of the 
above research strategies can be 
applied in most practical con- 
texts. In interview situations, 
items have content and the correct 
answer is almost always unknown. 
The item-reversal appr' 'ach has 
been devised for use in such cir- 
cumstances. If an item can be 
administered in two fprms so that 
the content is precisely reversed 
in the two administrations, self- 
contradictions in .which the re- 
spondent agrees with both state- 
ments are a clear indication of 
acquiescence. For example, if a 
respondent replies "yes" to both 
"I earn more than $5000 per year," 
and "I earn $5000 or less per year," 
response is clearly based on ques- 
tion form rather than on question 
content. This technique gives, 
the most definitive demonstra- 
tions of acquiescence, and v/as 
employed in the present study. 
Its only major difficulty is that 
clear content reversal of many 
items , especially complex atti- 
tudinal items, is virtually im- 
possible. Rorer's (1965) critique 
of item reversals with the F scale 
makes this point quite convincingly. 

Hare (1960) administered original 
and reversed forms of a 44-item 
interview with an "agree"-"dis- 
agree"-"can' t answer" response 
fprmat. S^ibjects were black and 
white womc'i. Self-contradiction 
was obser^yad, particularly among 
black women, who gave contradictory 
answers to an average of 20 of 
the 44 pairs of items. Hare did 
,not separately report rates for 
contradictions of the agree-agree 
and disagree-disagree types (a 



common flaw in item-reversal re- 
search) , but did report that con- 
tradictions were "mostly" of" the 
agree-agree type. This result in- • 
dicated that a tendency on the part 
of certain types of respondents 
to acquiesce can generally in- 
validate interview data. 

One study directly investigated 
the effects of acquiescence with 
mentally retarded respondenf-s . 
Gozali and Bialer (1968) , working 
with a personality inventory called 
Children's Locus of Control Scale, 
reversed the 23 items on this 
scale and administered both forms 
to a sample of 189 mentally retarded 
individuals of both sexes. Sub- 
jects ranged in age from 16 to 30 
years and in IQ from 58 to 91. 
The measure of locus of control 
derived from the standard form of 
this test was highly correlated 
with the measure of locus of control 
derived from the reversed form; 
that is, responding to the scale 
appeared to be based on item con- 
tent rather than item form. Un- 
fortunately, Gozali and Bialer ^ 
reported only correlation coeffi- 
cients, mixed questions keyed "yes" 
and "no" on both forms, cind did 
not report self-contradictions on 
an item-by-item basis , leaving us 
uncertain cibout the extent to 
which acquiescence might have been 
operating. 

Lenski cind Leggett (1960) imbedded 
a single question and its reversal. 
("It's hardly fair to bring chil- 
dren into the world, the way things 
look for the future," vs. "Children 
born today have a wonderful future 
to look foirward to,") in a long 
interview, separated by about 
one-half hour. For the sample as 
a whole, 8% of respondents agreed 
with both items. This tendency 
to acquiesce was related to both 
education and race. Overall, 20% 




of black versus 5% of white re- 
spondents agreed with both state- 
ments . For white respondents , 
only 2% of those with some college 
education acquiesced^ while 9% 
of those with 8th grade education 
or less acqiiiesced. Among black 
respondents , 14% of those with 
some college agreed with both 
items f compared to 32% of those 
with 8th grade education or less, 
Thus^ black respondents and less-, 
educated respondents showed a more 
marked tendency to acq\iiesce. 

Rothenberg (1969) used item- 
reversal in studies of cognitive 
development in yoxang children. 
Studying conservation of nianber^ 
Rothenberg asked his subjects 
both "Does this bunch have the same 
number of block? as this bunch?" 
and "Does this bunch have more 
blocks?" Subjects were 4- and 
5-year-olds , divided into lower 
and middle class (distinguished 
by neighborhood) . This class 
distinction also divided the 
sample according to race, for the 
lower class group consisted of 
black and Puerto Rican children, 
while the middle class sample 
included no Puerto Ricans and 
only 4% blacks. Additionally, 
the class distinction appears to 
have divided the sample according, 
to intelligence. As measured by 
the Peabody Picture Vocabulary 
Test, the 4- and 5-year-old lower 
class children had average IQ's 
of 72.2 and 72.5, respectively. 
The middle class 4- and 5-year-olds 
had IQ's of 110.6 and 101.9. On 
a single test of conservation, 
65.1% of lower class 4-year-olds 
contradicted themselves by answer- 
ing "y®s" to both forms of the 
question stated above. The ac- 
quiescence rate was 40.0% for lower 
class 5-year olds, 7% for middle 
class 4-year old5^ , and 10.0% for 
middle class 5-vear olds. The dif- 
ference between average acquies- 



cence for the lower middle class 
samples was significant at the .001 
level. Because race, social status, 
and IQ are confounded in this design, 
it is impossible to determine which 
of these variables was responsible 
for the difference in acquiescence 
rates in the two groups. However,- 
this is a clear demonstration 
of the potentially ruinous effects 
of acq\iiescence on validity. 
For the sample as a whole (N = 210) , 
27.8% of all responses were yes-yes 
o%lf-contradictions. (This far 
ouv. weighs the incidence of no-no . 
self-contradictions, at 5.2%.) 
The fact that this very high 
acq\iiescence rate occurred among 
youiig respondents and was especially 
pronounced for respondents of low ^ 
intelligence at least suggests 
that acqiiiescence may be a parti- 
cular problem in interviewing 
mentally retarded individuals. 

Measures of Individual Differences 
in Acquiescence 

A fourth approach to the eval- 
uation of acquiescence involves 
the construction of scales spe- 
cifically intended to measure the 
acquiescence tendencies of dif- 
ferent individuals. The develop- 
ment of such scales typically in- 
volves constructing a group of 
items which is held to be hetero- 
geneous with respect to. content. 
Furthermore, items are balanced 
so that equal numbers of agreeing 
and disagreeing responses are 
required for expression of an 
extreme preference within any 
content area. It is assxmied that 
the influence of content on the 
total number of agreement responses 
is thus eliminated, and the agree- 
ment score is taken to be a measure 
of inclination to acquiesce. In 
essence, this is simply an imprecise 
form of the item-reversal strategy. 
Couch and Keniston (1960) created 
such an acqtiiescence scale, con- 
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sisting of 360 items drawn from 
a variety of personality instru- 
ments and using a Likert type 
scale. Couch and Keniston attempted 
to correlate acquiescence as a 
personality variable with a number 
of other variables, constructing 
a. personality profile for the 
yeasaying respondent. For our 
purposes, their most significant 
finding was that yeasaying and 
intelligence are unrelated. How- 
ever, Couch and Keniston used 
crude measures of intelligence 
(the intelligent -dull scale of 
the 16 P.F., and the College En- 
trance Examination Board scores) 
and used a sample with a narrow 
range of intelligence (subjects 
were college undergraduates) . 
Thus, it is not surprising that 
yeasaying and intelligence were 
unrelated in this study. 

Wells (1963) derived a 20-item 
scale purported to measure ac- 
quiescent tendencies based on the 
earlier scale constructed by Couch 
and Keniston (1960). Wells re- 
ported that yeasayers, as defined 
by reponse to his scale , generally 
tend to endorse a wide variety of 
behavioral items. Further f Wells 
found that yeasayers tond to over^ 
report in surveys investigating 
such topics as possession of maga- 
zines and recall of advertisements. 
Finally, it is interesting to i.ote 
that Wells found yeasaying parti- 
cularly common among younger and 
less-educated respondents. 

It is tempting to interpret this 
latter finding as lending strong 
support to the findings of the item- - 
reversal literature. However, 
this individual differences research 
must be' interpreted very carefully; 
the imprecision of the content 
balancing procedures opens these 

scales to other interpretations. 

Inspection of the items of the Wells 



(1963) scale suggests that rather 
than being "content balanced," 
this scale may in fact test for 
an attitude of optimism or uncritical 
enthusiasm. The conclusions of the 
acquiescence scales literature 
cannot be readily generalized to 
the issue of acquiescence as a 
response set tending to invalidate 
interview data. 

Sum mary and Conclusions 

There is a striking body of 
evidence attesting to the fact that 
interview data can be invalidated 
by a tendency on the part of re- 
spondents to give agreement re- 
sponses independently of the eon- 
tent of the question. The studies 
using content-free measures suggest 
^a possible mechanism for this 
effect; when respondents lack more 
substantive bases for responding, 
they tend to prefer agreement re- 
sponses. The data drawn from 
situations where the correct answer 
is known support this conclusion 
and suggest additionally that an- 
^swers to false items are more valid 
than answers to true items. The 
most impressive data on acquiescence 
come from item-reversal studies. 
These studies demonstrate clearly 
that acquiescence can operate 
powerfully enough to seriously 
reduce the validity of interview 
responses. Also, item reversal 
studies suggest that black respon- 
dents, less-educated respondents, 
and less intelligent 'respondents 
may be particularly likely to 
acquiesce. Theae latter two sub- 
ject variables c.re, of course,^ of 
direct relevanca to our efforts 
to interview mentally retarded 
respondents. Finally, data from 
individual difference measures of 
acquiescence have som^ indirect 
relevance to the present study. In 
short, acquiescence is potentially 
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a very serious problem in inter- 
views, and the data suggest that 
the problem may be accentuated in 
interviews with mentally retarc'ed 
populations. 

Issues of Self-Presentation 

Many investigators have sug- 
gested that distortion of responses 
may occur when the respondent seeks 
to portray himself of herself in a 
particular way. Sudman and Bradburn 
(1974) refer to such response effects 
as arising from "issues of self 
presentation." within the personality 
assessment literature, much atten- 
tion has been given to the influ- 
ence of soci:±l desirability on re- 
sponding. That is, respondents may 
tend to endorse ite^s perceived as 
socailly desireable and to avoid 
endorsing items preceived as so- 
ei-ally undesirable, vithout re- 
guard to the "true answer." Thus., 
respcase patterns would reflect the 
perceived social desirability of 
the items, and not the actual behavr 
iors or attitudes of the respon- 
dents. Sudman and Bradburn (1974) also 
raise the possibility that items of 
an extremely personal nature might 
threaten the respondent and alter 
responses. 

Edwards (1957) presented data 
suggesting the importance of social 
desirability as a source of re- 
'sponse effects. Subjects' scores 
on personality scales frequently^ 
correlate very highly with the 
rated social desirability of scale 
items. Edwards also argued that 
the correlation between two measures 
frecjuently reflects the degree to 
which they have similar properties 
on the dimension of social desir- 
ability. 

Nevertheless, Sudman and Bradburn 
conclude that response effects 
attributed to threat and social 



desirability are generally small. 
The interaction of self-presenta- 
tion variables with other variables 
may, however, be important. For 
example, respondents are more likely 
to present themselves favorably when 
interviewed face-to-face than when 
afforded the relative privacy of 
a self-administered questionnaire. 
Thorndike, Hagen, and Kemper (1952) 
compared 500 self-administered and 
500 face-to-face administrations of 
an inventory on psychosomatic symp- 
toms. The respondents to the self- 
administered form reported some 15% 
more psychosomatic symptoms than did 
respondents to the face-tp-face in- 
terview. When Knudsen, Pope, and Irish 
(1967) compared women's responses to 
face-to-face and self-administered 
interviews regarding the permissibil- 
ity of premarital sex, 20% of respon- 
dents to face-to-face interviews' said 
it was all right to have premarital sex, 
as -opposed to 31% of the respondents 
to tHe self-administered form. In 
general, the magnitude of the effec:t 
due to method of administration in- 
creases as the probability of a so- 
cially desirable answer becomes 
greater. 

As has' been mentioned already, ques- 
tion structure also interacts with . 
self-presentation issues to produce 
response effects. Closed-ended ques- 
tions appear to increase response 
effects due to threat or social de- 
sirability. Furthermore, some inter- 
action occurs between social desir- 
ability and respondent and inter- 
viewer characteristics. Sudman and 
Bradburn (1974) found response effects to 
differ as a function of the race and 
sex of both interviewer and respond- 
ent, depending on the degree to 
which an interview activated concern 
with self -presentation. 

In short, issues of self -presentation 
in themselves have small response 
effects, but interact with other 
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variables to produce important re- 
sponse effects. ■ In particular, 
method of administration interacts 
with social desirability; face-to- 
face interviews involving a strong 
social desirability component re- 
sult in considerably less admission 
of .information placing the respon- 
dent in a negative light. This find- 
ing* may be important in interviews 
with mentally retarded persons, since 
these interviews are of necessity 
face-to-face. Mentally retarded re- 
spondents may tend to report atti- 
tudes and behaviors which they be- 
lieve will meet approval, especially 
if they perceive the interviewer as 
having power. Social desirability 
also interacts to some degree with 
quest.! on structure and with the race 
and sex of the interviewer' and re- 
spondent • 

Behavioral vs. Attitudinal 
Infonuation 

One can distinguish two types of 
information sought in interviews. 
The first type is behavioral infor- 
mation, sought through such questions 
as, "In what year were you born?" 
The distinguishing characteristic of 
such information is that it can in 
principle be verified by reference 
to outside sources. Attitudinal in- 
formation, on the other hand, in- 
volves the subjective valuations of 
the individual (e.g., "Do you like 
your current job?) . Such questions 
have no external referent; that is, 
they cannot, even in principle, be 
verified. 

There is some suggestion in the 
survey literature that attitudinal 
information is more subject to re- 
sponse effects than is behavioral 
information. However, because at- 
titudinal information has no exter- 
nal standard of accuracy, the mea- 
surement of response effects for 
behavioral and attitudinal 



information differs fundamentally. 
Thus, response effects for the two 
types^of questions cannot be di- 
rectly, quantitatively compared, and 
no general conclusion as to the 
relative susceptibility of the two 
types of questions to response 
effects is easily drawn. 

Availability of Information 

The availability to the indi- 
vidual of the Information sought 
may have powerful effects on re- 
sponse validity. That is, if in- 
formation is not very salient or is 
for some reason difficult to recall, 
the respondent's answer is likely 
to be biased. 

Sudman and Bradburn (1974) showed ' 
that response effects differ greatly 
according to the length of the recall 
period involved in reporting behav- 
ioral information. Respondents tend 
to over report behaviors for short 
recall periods and to underreport 
as length of recall increases. 
Length of recall period also inter- 
acts with. several other factors. 
Response effects increase substan- 
tially for individuals over 55 as 
the length of recall increases. 
Face-to-face interviews typically 
reduce response errors due to poor 
recall by reducing omission errors, 
although face-to-face interviews 
may also stimulate over reporting. 
Threatening topics increase response 
effects for longer recall periods 
due to omission. Also, open-ended, 
questions have been found to yield 
somewhat' weaker response effects 
over increasing recall periods 
than do closed-ended questions, 

Saliency of the information re- 
quested might be expected to- have 
an important effect on responses. 
Surprisingly, Sudman and Bradburn 
found that saliency of questions 
had no general effect on behavioral 
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items. However, low saliency in- 
^creased response effects for atti- 
tudinal items. The authors specu- 
lated that the difficulty of mea- 
suring saliency might have been re- 
sponsible for the lack of a strong 
effect. 

In short, long recall periods in- 
crease response effects. Length 
of recall interacts with several 
other task variables in producing 
this effect. No general effect 
due to saliency of information 
was observed, although low sali- 
ency increases response effects for 
attitudinal items. 

Methou of Administration 

Analysis of response effects 
due to method of administration 
essentially involves a comparison 
of face-to-face with self-adminis- 
tration. Those effects are gen- 
erally best int_rpreced 'as arising 
from inter^actipn with other task 
chatacteristics. Most notably, 
as mentioned above, respondents 
are- far more willing on self -ad- 
ministered interviews to give in- 
formation placing them in an unfavor- 
able light. In fact, most of 
the response effects attributable 
to method o-f administration pro- 
bably arise from the fact that face- 
to-face interviews intensify fbr the 
respondent those issues of self-pre- 
presentation raised by the content 
of the interview. 

INTERVIEW VARIABLES 

The biasing effect of inter- 
viewer characteristics on survey 
responding is the most widely re- 
searched area in the response ef- 
fects literature. Far more effort 
has been devoted to this topic than 
to the assessment of response ef- 
fects due to task variables. Never- 
theless, Sudman and Bradburn (1974) 



conclude that response effects due 
to interviewer characteristics are 
generally less important than those 
due to task variables. When inter- 
viewer vari,ables exert important ef- 
fects, it is usually in the context 
of highly specific relationships 
among interviewer characteristics, 
respondent characteristics, and the 
topic of the interview. 

Sudman and Bradburn also found that 
the social status of the interviewer 
could interact with task and respon- 
dent variables to engender response 
effects. Katz.(1942) for example, 
found that working-class inter- 
viewers obtained more pro-labor 
responses, particularly from union 
members, than did middle-class in- 
terviewers. On war issues, middle- 
class interviewers obtained more 
interventionist responses than did 
working-class interviewers. The 
finding may be of importance in 
interviewing mentally retarded per- 
sons, particularly where the respon- 
dent might regard the interviewer 
as having authority. 

Sudman and Bradburn (1974) found no 
general effect due to sex or race of 
the interviewer, or /to any other comr 
bination of interviewer character- 
istics. The only exception to 
this general lack of interviewer 
effects pertains to "don't know" 
answers. Fragmentary evidence sug- 
gests that interviewers over 50 
get more "don't know" responses, 
that interviewers of higher social 
class and with more experience get 
fewer "don't know" responses, and 
that the "don't know" rate may be 
higher for female interviewers. 

Interactions of Interviewer 
Characteristics with Other Variables 

There are .some* interactions be- 
tween interviewer characteristics 
and method of administration, 
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question structure, and self- 
presentation issues. 

Jhe.most notable^ response effects 
attributable to interviewer vari-^ 
ables occur when there are specific 
combinations of interviewer and re-- 
spondent characteristics and topic 
of interview. For example, Schuman 
and Converse (1971) conducted 500 
interviews in black households, 
using both black and white inter- 
viewers. Response effects due to 
interviewer race were large for 
items having a definite racial con- 
tentr especially items involving 
militancy cr hostility toward whites. 
In response to the question "Do you 
personally feel you can trust most 
white people, some white people, or 
none at all?", 35% of respondents 
replied "trust most whites"* when in- 
terviewed by a white person, as op- 
posed to only 7% when asked by a 
black interviewer. Schuman and Con- 
verse observed little or no effect 
due to interviewer race on questions 
not involving militancy ort hostility 
to whites. This illustrates the 
general conclusion that response 
effects attributed to interviewer 
race or sex are highly specific to 
the interview topic. 

In STommary, larger response effects 
tend to occur when the interviewer 
is young or inexperienced. The most 
notable response effects occur, how- 
ever , when the topic of the inter- 
view is such that some combination of 
interviewer and respondent character- 
istics may bias respondent's answers. 

RESPONDENT VARIABLES 

It seems likely that some types 
of respondents are more prone than 
others to giving biased answers. 
Sudmand and Bradburn (1974) found no 
general response effects associated 
with the sex, race, or age of the 
respondents. They did find that the 



largest response effects tend to 
occur for respondents with eight 
years or less of school. This re- 
sult does not refer to poorly edu- 
cated adults but to children who 
are still in school. It is not clear 
what characteristics of school chil- 
dren make them ^ specially ^ sceptible 
to response effects, buc t,i±s finding 
may have important implications for 
interviewing mentally retarded per- 
sons. Mentally retarded individuals 
are both "young" in the sense of men- 
tal age and uneducated, in that they 
do not progress normally through the 
educational system. Th|^s, if it is 
either low mente.l age or lack of edu- 
cation that exaggerates response ef- 
fects in school children, we may ex- 
pect that mentally retarded indivi- 
duals will be particularly suscep- 
tible to response effects. 

'Sudman and Bradburn also report some 
response effects due to particular 
combinations of respondent and task 
variables. Respondent variables in- 
teract to a minor extent with level 
of threat, method of administration, 
structure of questions, length of 
questions, and the possibility of 
a socially desirable answer. Two 
respondent-task-interactions, already 
discussed under task variables, are 
of particular relevance to our re- 
search. First, response effects for 
closed-ended questions (i.e., over- 
reporting) are particularly high for 
elementary school students. Second, 
various response effects engendered 
by long questions are particularly 
strong for respondents with high 
school education or less. 

In short, elementary school students 
are especially susceptible to re- 
sponse effects. This effect is in- 
creased by the use- of closed-ended 
questions. Further, less educated 
respondents tend be more strongly 
affected by long questions. 
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Implications for Interviewing 
Retarded Persons 

What do these two very different 
bodies of literature^ one on language 
and communication skills of the men-' 
tally retarded and the other on re- 
sponse effects in survey research, 
tell us about what to expect when we 
interview retarded individuals? 
Although we have discussed specific 
implications along the way, we would 
like to emphasize the following ob- 
servations here: 

1. There has been almost no re- 
search, prior to ours, specifi- 
cally examining the performance 

• of retarded persons in inter- _ 
view situations. Thus it must 
be recognized that the litera- 
ture we have reviewed here is 
only indirectly relevant to our 
questions of interest and that 
any expectations based on that 
literature may or may not be 
borne out when retarded indi- 
viduals are actually inters 
viewed. 

2. As a general rule, we can ex- 
pect retarded persons to behava 
linguistically much like 
younger persons of normal intel- 
lectual functioning. For the 
most part, they can be expected 
to perform like children of com- 
parable mental age. However, at 
times they may perform below the 
level one would expect on the 
basis of mental age, and occa- 
sionally (e.g., with respect to 
vocabulary size) they may exceed 
such expectations. Such delays 

* in language development repre- 
sent quantitative differences 
between retarded and normal-IQ 
individuals, and thus no need 
to shape interviews in^ accord 
with unique characteristics 
of retarded persons. 



3. Because language development is 
generally related to mental age, 
little can be expected of re- 
tarded adults who are profound- 
ly retarded or retarded chil- 
dren v?ho have very low mental 
ages. Certainly interviewing 
would appear to be unfeasible 
with persoxis whose mental ages 
are below three? on the other 
hand, answering questions 
appears to be well within the 
cognitive and linguistic capac- 
ities of those with mental ages 
above seven or eight. While 
mental age can be viewed as . 
the most important considera- 
tion, other factors may also 
influence how well an individ- 
ual can handle the demands of 
an interview. For example, 
expectations may be somewhat 
lower for Down ' s Syndrome chil- 
dren and for those who are in- 
stitutionalized . 

4. Another factor with major im- 
plications for interviewing 
retarded persons is their high 
rate of speech and. hearing dis- 
orders. Even when mental age 
is relatively high, speech 
defects and disorders may, make 
it difficult for the interviewer 
to understand what the inter- 
viewee is attempting to say. 
More crucially, the high preva- 

* lence of hearing disorders 
among the retarded suggests 
that some screening for hearing 
defects might be necessary or 

■ that alternative (nonverbal) 
communication systems might be 
necessary with persons who have 
significant hearing losses. 

5. Studies of semantic development 
among the mentally retarded 
point" to the need for care in 
wording questions. Most 
notably, concrete wording 
should be favored over abstract 
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wording, and commonly used words 
should be preferred over infre- 
quently used words. Studies of 
the cognitive development of 
young children can serve as a 
useful guide in deciding what 
types of concepts might be too 
cognitive ly advanced for many 
retarded persons. 

Similarly, the structure of 
questions asked of retarded 
persons should be kept as simple 
as possible. Questions with 
dependent clauses, passive con- . 
structions, negatives , and so, 
because they are transforma- 
tionally complex, can be ex- 
pected to' be difficult to 
understand. To be preferred 
are questions that are based 
on simple active declarative . 
"kernel" structures and that are 
as short as possible. Studies 
of question comprehension and 
production among normal children 
suggest a regular developmental 
sequence that can serve as a 
guide to constructing questions. 
For example, yes-no, what, and 
where questions can be expecjted 
to be easiest for retarded 
persons to understand and 
answer, while who, how, and 
why questions, and especially 
when questions, appear to re- 
quire a higher level of cogni- 
tive development.- 

7. Retarded individuals, like nor- 
mal children, can generally be 
expected to understand more 
advanced linguistic foinns than 
they can produce on their own. 
As we have noted, the implica- 
tion of this for designing 
interviews is that structured 
question fomats may prove 
easier to answer than open-ended 
formats because they reduce the 
demands on the interviewee to 
compose and express lengthy 



answers. Questions should still 
be kept as simple as posisible to 
aid comprehension, but anything 
that would make answering easier 
seems particularly advisable 
(e.g., asking respondents to 
point to a response, give a 
single word answer like "yes" 
or "no," or select one of two 
or three options) . 

8. Whereas the_ literature on lan- 
guage and communication among 
the mentally retarded suggests 
that they may have difficulty 
understanding and answering 
questions, the literature on- 
response effects in survey 
research with the general popu- 
lation suggests that what they 
say may not be valid. This re- 
search has identified nxamerous 
-* ways in which answers are af- 
fected by the. nature of the 
interview task or the Specific 
questions asked, charactertistics 
of the interviewer, and charac- 
teristics of the respondent. 
There is no reason to believe 
that, mentally retarded inter- 
viewees are immune from such 
effects. Indeed, there is 
reason to believe that their 
answers might be especially 
influenced by such factors as 
how a question is worded or 
whether it is closed-ended or 
open-ended, for less educated 
members of the general popula- 
tion as well as young children 
have been shown to be especially 
susceptible to response effects. 

9. The response effects literature 
does not always suggest that 
one way of asking a question is 
preferable to another; often 
studies only demonstrate dis- 
.crepancies between the answers 
that are given. However, this 
research does suggest the wis- 
dom of avoiding questions that 
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contain leading or emotionally 
laden wording; that are unusu- 
ally long and complex; and that 
encourage respondents to pre- 
sent themselves in a socially 
desirable manner. Acquies- 
cence^ or the tendency to agree 
regardless of question content, 
would appear to be an especial- 
ly dangerous threat to the 
validity of answers given by 
retarded persons. 

Finally, the literature on 
response effects in survey 
research reminds us that any 
interview is a social situation 
affected by characteristics and 
behaviors of both the inter- 
viewee and interviewer. The 
answers given by mentally 
retarded persons may be 



affected not only by their 
characteristics but also by 
such factors as the inter- 
viewer 's age cr the amouir:: of 
experience zr ?. ic has. 
Moreover , nziBeroi: 
time 2 in i ^e , ::he r 

may 3 c~ . tciB..., 
inte viev anc per. x 
characterx^ ..xcs tuac axe es- 
pecially likely to yield in- 
valid answers. While our 
research did not examine the 
effects of interviewer charac- 
teristics, we did look care- 
fully at response biases in 
the answers given by retarded 
interviewees, attempting' to 
determine how both the way the 
question is asked and the IQ 
of the respondent affect the 
validity of what is said. 
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Chapter 3 



METHODS OF THE STUDY 



The project was designed to ad- 
dress the following questions about 
interviewing retarded persons: 

1. To what extent can retarded 
persons respond to questions 
in an appropriate fashion r 
and what factors affect their 
responsiveness? 

2. How reliable are such respon- 
ses r in the sense of being 
consistent over short periods 
of time? 

• 3. How valid are such responses, 
in the sense of being free of 

' systematic biases and agree- 
ing with information provid- 
ed by parents or caretakers 
or documented in records? 

I 4. What types of questions 

I appear to optimize respon- 

! siveness, reliability, and 

I validity? 

These guiding questions were explored 
tlirough a series of five studies, 
each focused on a subsample of the 



retarded population and each having 
its own priorities, and basic, design. 
After describing the process of 
questionnaire development, we will 
provide an overview of the individual 
studies and of the methods employed 
in all of them. 

Design of Interview Schedules 

In early discussion ^with repre- 
sentatives of the President's 
Committee on Mental Retardation, the 
scope of topics to be asked about was 
delimited. Three major topics were 
identified: living circvimstances, 
familiarity with and utilization of 
services, and opportunities for 
decision-making. As a result, the 
interview schedules initially con- 
structed typically included questions 
about basic characteristics of the 
respondent, major day activities, 
extent of involvement in other ac- 
tivities, nature of the residential* 
setting, training currently being 
received, and independence in daily 
decision-making. 
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Interview schedules used, previously, 
including ones used in the Center's 
own studies of deinstitutionaliza- 
tion, were reviewed to generate 
potentially useful questions and 
ideas for alternative question phras- 
ing. Pilot interviews were devel- 
oped during the first part of 1977, 
and they were tested on a small 
sample of adults, both Institution- 
alized and living in the local com- 
munity, in May, 1977. Extensive 
revisions and analyses of individual 
items were then conducted, and the 
pilot interview schedule was sent to 
PCMR for review. After further re- 
finement, questionnaires were con- 
structed for use in the first 
interviewing venture with institu- 
tionalized children and adults in 
August, 1977. In these studies, 
which addressed the reliability 
question, some questions were simply 
repeated in exactly the same form 
on two interview schedules adminis- 
tered six to eight days apart. In 
addition, in these studies as well as 
in later ones, a major effort was 
made to include alternatively struc- 
tured or worded question^ on the 
same topics in order to test the 
relative merits of various ques- 
tioning strategies. For example, in 
asking about decision-making, we 
tested these two alternatives: "Who 
decides what clothes you put on in 
the morning?" versus "Is it up to 
you what clothes you put on in 'the 
morning?" 

It should be noted that while some 
effort went into establishing the 
content of the interview schedules , 
it was minor in comparison to the 
effort that went into developing 
, workable questions and alternative 
question foinns. In other words, we 
were not overly concerned with devel- 
oping a standard protocol to fully 
address the questions of living j 
circiamstances , services , and 1 



decision-making. Instead, while 
using these topic areas as a context, 
we concerned ourselves primarily 
with exploring feasible interviewing 
strategies. On occasion, we included 
questions which we expected to 
fail simply to document that certain 
approaches are relatively unworkable. 
On occasion, too, we deviated from 
the three major topic areas in our 
.. quest for understanding of interview 
behavior. Finally, we did a con- 
siderable amount of alteration of 
questions from study to study, re- 
fining questions that had been used 
before and introducing new questions 
appropriate to the samples under 
study, so that, in the end, the 
various samples we interviewed 
responded to very few of the same - 
questions. Appendix A, by presenting 
many of the questions that were in- 
cluded in the various interview 
schedules, offers a more concrete 
illustration of the kinds of ques- 
tions which were asked. 

The Five Study Samples 

Because the study was explora- 
tory in nature, its design evolved 
over time, but from the start there 
was an interest in tasting the feasi- 
- bility of interviewing with different 
segments of the mentally retarded 
population. The five studies col- 
lectively involved both children 
and adults, in both institutional 
and community settings, as, well as 
identified significant others 
(parents, attendants, and advocates). 
In all, 180 mentally retarded persons 
were interviewed. Since some parti- 
cipants were administered repeated 
interviews, the total number of 
client interviews held was close to 
320, and in addition, 210 interviews 
were conducted with significant 
others. The five studies and their 
samples are described on the following 
page . 
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V 



INSTITUTJON CHIIDREN 

The study of institutionalized 
children focused on 52 children, age 
12-16, who were residents of a sin- 
gle state institution for the retard- 
ed:. 20 severely retarded by 
Stanford-Binet cutoffs (IQ 20-35), 
16 moderately retarded (IQ 36-51), 
and 16 mildly retarded (IQ 52-68). 
A list of eligible participants was 
drawn by random sampling of the 
facility's files, and provision of ^ 
consent and availability for inter- 
view then determined which eligible 
subjects (those in the right age and 
IQ range) were actually interviewed. 
Two interview forms were used, with 
some questions repeated and others 
phrased differently on the two forms, 
and with the forms administered 
approximately a week apart. Through 
counter-balancing, half of the 
children received one form first 
and half received the other, half 
. were interviewed by one interviewer 
first and half by the second, and 
each child was then administered the 
other form by the other interviewer, 
as shoWn in Table 3,1. A cottage 
attendant familiar with each subject 
was also interviewed using an inter- 
view schedule which included most 
of the same questions asked of res- 
idents (with "you" changed to " he" 
or "she"). , 



INSTITUTION ADULTS 

The study of institutionalized 
adults was identical in design, but 
involved adult residents of the same 
institution, including profoundly 
retarded persons. A total of 58 
adult subjects were ultimately 
interviewed: 16 profoundly retarded, 
16 severely retarded, 16 moderately 
retarded, and 10 mildly retarded. 
The interview schedules used with 
this, group were longer and more 
intricate than those used with 
any other groups, and again 
attendants were asked most of the 
same questions. 

COMM.' KTY CHILDREN 

Tne third study focused on 
children in the 12-16 age range; 57 ' 
in all, 18 severely retarded, 19 
moderately retarded^ and 20 mildly 
retarded; none of whom were institu- 
tionalized. Use of the 12-16 age 
range in both this Scimple and the 
institution children sample made 
them comparable in approximate mental 
age (the five- to eight-year old 
range) . Students at the lower IQ 
levels, were drawn from a school for 
the trainable mentally retarded, 
while mildly retarded subjects were 
drawn from other special education 
programs in the same city. Informed 
consent from parents, obtained by 



Table 3.1: Study Design 



1st Administration 


2nd Administration 




i 


of 


sample : 


Form, A, Interviewer 


1 


Form B, Interviewer 


2 




i 


of 


sample: 


Form A, Interviewer 


2 


Form B, Interviewer 


1 




i 


of 


sample : 


Form B, Interviewer 


1 


Form A, Interviewer 


2 




i 


of 


sample : 


Form B, Interviewer 


2 


Form A, Interviewer 


1 
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home delivery of forms by children, 
determined which eligible siibjects . 
were in the final sample / thus 
making -it impossible to determine 
the extent to which the sample was 
representative. This study did not 
deai with the interviewer- re interview 
reliability issue, but instead tested 
several new questions appropriate to 
community residents and several 
alternative question formats on the 
same topic. Each subject's parent 
or guardian was also interviewed. 

The original design of the project 
called for extending the research to 
a fourth major group of interest: 
adults in the community. However, 
by th time that final phase was 
being planned, we had had an oppor- 
tunity to examine data from the 
first three samples and decided to 
alter the plan. It had become 
apparent that interviewing severely 
retarded persons and interviewing 
more highly verbal, mildly retarded 
persons were two different under- 
takings. On the one hand, we still 
wished to interview adults living in 
the community but felt that it might 
be possible with such a group to t2 y 
out more unstructured questions and 
complex topics. On the other hand, 
we felt a need to search further 
for techniques which could be suc- 
cessfully employed with low verbal 
•persons. As a result, two additional 
studies, far different in samples and 
methods were conducted. 

COMMUNITY ADULTS 

To. facilitate identifying a di- 
verse sample of adults living in the 
community, we made qontact with>\the 
Association for Retarded Citizens in 
another city in Texas and requested 
their cooperation in the interviewing 
study. Letters and informed consent 
forms were mailed to. ARC-based citi- 
zen advocates, who were to indicate 
whether^ they would be interested in 
being interviewed themselves. They 



were alfeo asked to explain the proj- 
ect to their proteges and determine 
if they too would be v;illing to be ^ 
interviewed. Unfortunately, of the 
70 some letters mailed, very few were 
returned, and even with follow-up 
phone calls informed consent was ob- 
tained and interviews could be 
scheduled with only 13 mentally 
retarded participants arid their 13 
advocates. The retarded participants, 
whose exact IQ scores weie not 
obtained, were generally^ in the mild 
or moderate range of retardation and 
represented a highly verbal group. 
Interviews with them were conducted 
by a single interviewer and were 
tape recorded. The study, which 
yielded long and enlightening tran- 
scripts, provided an opportunity to 
use unstructured probing techniques 
and to explore difficult areas such 
as satisifaction with services and 
reactions to being, or beintj con- 
sidered retarded. While we will 
not provide detailed findings in 
view of the small sample size, we 
will occasionally quote from the 
transcripts to illustrate processes 
and problems in interviewing. 

RE INTERVIEWING OF LOW VERBAL CLIENTS 

The final study in the series 
was designed to further explore 
techniques of eliciting information 
from low verbal subjects. It in- 
volved re interviewing institution- 
alized children and adults who had 
been largely, but not totally, un- 
responsive in the previous institu- 
tion studies. The 29 subjects ranged 
in IQ from 20 to 48 and were in the 
lower half of the distribution of 
responsiveness, but had been able to 
answer at least one question in the 
first interviews. The phenomenon of 
acquiescence was explored further in 
this study and new approaches using • 
pictures were tested out. Charac- . 
teristics of the samples are sum- 
marized in Table 3.2. 
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than the procedures we actually used. 
To compose the institution samples, 
an approach was made to a facility in 
Arkansar with which the Center had 
had pr:. '.dus nelationships. A pro- 
posal was submitted for review and 
approval by the body charged with 
determining the appropriateness of 
research proposals affecting clients 
in the Arkansas MR Services system. 
Although it did not at first appear 
that consent from all parents would 
be necessary/ a decision was made 
that such consent would be necessary. 
As a result, a trip that was to have 
been the data collection trip turned 



Table 3,2: Characteristics of Samples 





IQ 

M 


SD 


Age 

M- SD 


Institution Children 
(N = 52) 


42.08 


13.81 


14,81 


1,25 


Institution adults 
(N = 58)* 


39,76 


13.12 


23.49 


4.42 


Community children 


47.53 


12.12 


14,03 


1.92 


(N = 57) 










Community adults 










(N = 13) 










Reinterviewed institution 










sample 


31,62. 


7,24 


19.58 


4.76 


= 29) -^^ 











Access to Subjects 

since gaining permission to con- 
duct a study and gaining consent 
from potential participants are part 
of any interviewing study, a word on 
the procedures used in the present 
study is in order. We elected to 
make contact with particular agencies 
or facilities, each of which had its 
own procedures which we were then 
obligated to follow. If we had been 
seeking a representative sample of 
mentally retarded persons of all 
ages and circumstances, the proce- 
dures used would" have been far more 
elaborate and difficult to implement 



*The entire adult sample consisted of 58 persons, 16 of them with IQs from 0-19. However, 
as they were generally uninterviewable, and as most analysis focuses on the seve'^e through 
mild ranges of retardation, they are not included in these figures, which are instead based on,, 
42 participants in the severe to mild range of retardation. 




into a lengthy process of gaining 
consent. Parents of children and 
adults potentially eligible for the 
study were called by project staff 
members in the presence of a witness 
and asked to provide their consent. 
Following this, letters were sent 
to those parents or guardians who 
had given consent verbally so that 
their consent could be provided in 
writing. In addition, each client's 
verbal consent was obtained before 
the interview was actually begun. 

In the' study of community children, 
the planned research again had to 
be approved through normel mechanisms 
in the school system. In this case, 
there was a prohibition against the 
research project staff making direct 
contact with parents or children, so 
a designated staff member in the cen- 
tral administration was responsible 
for distributing explanation letters 
and consent forms to teachers of 
mentally retarded students in the 
specified age range and then collect- 
ing the forms which found their way 
home witlj the students, and were 
returned. Using this procedure, many 
forms were never returned, and when 
it became apparent that few mildly 
retarded subjects were being identi- 
fied, new rounds of forms were dis- 
tributed to teachers of the mildly 
retarded". 

Similarly, to constitute a community 
adult sample, we first had to obtain 
the endorsement of a local Associa- 
tion of Retarded Citizens. In this 
case, a project staff member was 
given permission to mail letters and 
forms 'directly to all citizen advo- 
cates and to directly receive the 
returns, and written client consent 
was also required. Again, many form^ 
were not returned. In no study 'was 
the rate of refusal high, but in 
those that involved return mailing 
of consent forms , low rate of return 
was the general rule. Unfortunately,' 
this creates uncertainty about how 
representative those who actively 



provide consent are. The telephoning 
procedure used to constitute the 
institution samples was highly effec- 
tive by comparison, but in view of 
the recent tightening of procedures 
for access to subjects, it may not 
continue to be acceptable, for i't 
does involve a. f acili .ry ' s making 
available to outside researchers 
clients' names and addresses. The 
moral is that an interviewing project . 
of this nature must leave adequate 
time for obtaining permission to 
conduct the study and for gaining 
consent. There are special problems 
in making the nature of the study 
clear to mentally retarded people. 
Appendix B discusses some of the 
current issues in obtaining research 
approval and informed consent. 
Appendix C illustrates how the 
present study was explained to men- 
tally retarded persons immediately 
before they participated and in- 
cludes a sample consent letter to 
parents. 

Measuring Responsiveness 

An important focus of the project 
throughout was the ability of men- 
tally retarded persons to answer 
questions in a manner appropriate 
"to the form and content of the ques- 
tion, whether or not the answer was 
"true" or valid. Responsiveness to 
the communication demands of ques- 
tions was measured in each study 
through a slight modification of a 
coding system originally developed 
by Sigelman and VJerder (1975) , which 
included nine categories: 

1. no response 

2. unintelligible response 

3. irrelevant response 

4. don't know, don't remember, 
not sure (when respondent can 
be presiamed to know) 

5. inadequate response (vague or 
uncodable) 
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6. request for clarification of the 
question (huh? response) 

7. refusal to answer 

8. minimally appropriate response 
(response which meets the formal 
demands of the question by pro- 
viding one relevant bit of in- 
formation, e.g., "yes" or nam- 
ing one item in response to an 
open-ended question) 

9. expanded response (response which 
provides qualifying or additional 
information beyond the formal de- 
mands of the question) 

Interrater reliability for this coding 
system has consistently been estab- 
lished at .85 or higher. In all studies, 
the interviewee's first response to the 
question was coded in this manner by 
the interviewer either during the inter- 
view or afterwards usir-g notes on the 
response given. In the adult institu- 
j tionai sample, responses to a second 
1 asking of the question when the first 
I response was inappropriate were also 
Icoded in this manner. In the children's 
^institutional sample questions were 
jasked only once, while in all other 
Scimples questions were asked a second 
time, if necessary, in exactly the 
same foxm. While .responsiveness to 
l^he first asking formed the measure of 
responsiveness in all studies, . odings 
d£ the contents of responses were bas- 
est on whichever response, first or sec- 
pAd, was at least minimally appropriate. 
The number of responses coded 8 or 9 
(n^inimal or expanded) was divided by 
th^ number of questions asked to form 
individual responsiveness scores. To 
form difficulty scores for given ques- 
tions, the number cf interviewees whose 
responses were coded 8 or 9 was divided 
by the number of persons asked the ques- 
tion. 

' In all studies a simple screening pro- 
cedure was used when a subject's abili- 
ty t^o engage in an interview was sus- 
pect;. The procedure consisted of three 



simple questions ("What is your name?", 
"Is your name (name)?", and "Can you un- 
derstand me?) , each of which was asked 
a second time. if no initial response 
was obtained. If any response (word, 
sound, gesture) to one of these questions 
was qiven, a subject was administered 
the full interview. Those who would 
not respond at all were defined as to- 
tally unresponsive to all questions in 
the full interview, even though it was 
not actually administered. 

Analyzing Reliability and Validity 

In all studies contingency tables 
were analyzed to examine issues of 
reliability and validity. In compar- 
isons of subjects' responses to a 
first and second asking of the same 
question or to alternatively phrased 
questions on the's^e topic, contin- 
gency tables indicated the percentage 
of the re^;:^'^nding sample providing 
the same answers on both occasions. 
Similarly, subjects' responses can be 
cross-tabulated with significant 
others* responses to the same ques- 
tions. At the same time, cells 
representing inconsistency between 
paired responses can be examined to 
identify particular patterns of 
response error. The sample contin- 
gency table presented in Table 3; 3 
will give an illustration of the 
method used. It presents proportions 
(rather than actual nvjnbers) of 
various pairings of client and 
significant other responses about 
church attendance. If we look at 

the marginals of the table^ we would 

note that there are discrepancies 
between client and . significant other 
responses, for 80% of the clients 
claim that they go to church while 
70% of the significant others claim 
that clients go to church* If we 
add the proportions of yes-yes and 
no-no combinations of response, we 
have an overall agreement figure 
indicating that in 70% of the cases 
in which answers were paired, client 
and significant other agree. Ex- 
amining the cells representing dis- 



Table 3.3: Sample Contingency Table 



Significant 


other: 


Do 


you go to church? 






Does client 
church? 


g*/ to 


Yes 


No 






Yes 


.60 




.10 


.70 






No 


.20 




.10 


.30 








.80 


.20 







a9. eement, we note that it is more 
common for the client to say "yes" 
and the significant other "no" than 
for the client to say "no'* and the 
significant other "yes." In this 
instance, then, we have detected a 
systematic bias in responses that 
accounts for disagreement - one which 
probably reflects the acquiescence 
phenomenon vvfith which we will con- 
cern ourselves later. 

On the face of it, 70% agreement 
sounds fairly good. However, note 
that both clients and significant 
others were highly likely to answer 
"yes" to thi question, and that ^as a 
result, the ids by chance alone 
of obtaining many yes-yes combina- 
tions of answers is rather high. In 
other words, in tables like this one, 
we l^anno€"s~r^ 

chance alone client and significant 
others would have agrp.ed 50% of the 
time. Unfortunately, traditional 
statistical technig les desi^r- i^or 
use with contingency tables, ;viea?;u:-:es 
of strength of association .^x V 
prii and gamma and lambda ana mec^ouies 
of statistical significance such as 
chi-squarBf do not provide an^ appro- 
priate means of determining how much 
a given agreement figure exceeds 



what can be expected on tho basis of 
chance. However, Cohen (1960) has 
described a statistic called kappa 
which appears to suit this problem 
perfectly. ^ 

Because the project has involved 
hundredj of contingency tables, we 
have not calculated kappas for all 
tables, but we will make reference 
at various places in the report to 
kappa statistics, for they provide 
us with a 'means of putting agreement 
figures in some perspective. The 
same logic that we have described 
here also applies if one is concerned 
with whether clients answer the same 
question the same way on two occasions 
and whether their answers to two 
different forms of question are 
consistent. 
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Footnote 

^To illustrate kappa, note that the obtained, proportion of agreement for Table 3-3 
is. .70, The proportion expected by chance is easily calculated from the marginals associated 
with the two cells representing agreement- The chance probability of a yes-yes combination 
is .56 (.70 X .80), while the chance probability of no-no combination is .06 (.30 X .20). 
Therefore the proportion of agreements" expected by chance is .56 + .06 or .62, quite a bit . 
higher than 50-50. Kappa is calculated as tollows: 



k = 

In our example, 

k = 



p obtained - p by chance 
1 - p by chance 



.70 - .62 



.38 = .21 



Very simply, this means that 21% of the cases- represent agreement beyond agreement that one 
would expect on the basis of chance. Kappa ranges from -1.00 to +1.00, a negative kappa indi- 
cating that disagreement is more likely than one would expect, a kappa around zero indicatinn 
that agreement is not very 'different from what would be expected based on chance, and only a 
large positive kappa indicating agreement beyond chance probability. The statistical 
significance of a kappa can be determined by first calculating a standard error term. 



o ko 



Pc 



N(l - Pc) 

or in the example, assuming an N of 50, 



' .62 = .0326 = .18 

50(1 - .62) 



By referring to a normal curve distribution table, we find that the probability of this z_ value 
is .12, approaching the conventional .05 level of probability but not large enough to be 
considered statistically significant or a systematic departure from chance probabilities. 
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Chapter 4 



INTERVIEWING RETARDED PERSONS: LESSONS 

FROM EXPERIENCE 



Most of the suJDsequent chapters 
of this monograph coricern the inter- 
view responses of mentally retarded 
persons and their parents or care- 
^'takers — the data of the project. 
However, we have gained, through 
year's of experience interviewing 
mentally retarded pf>rsdns, another 
type of "data," often subjective in 
nature, about the process of con- 
ducting interview research with the ^ 
mentally retarded. Since mentally 
retarded people are, after all, 
people, much of wh^t we would have 
to say is simply a matter of sound 
survey research design and practice. 
The literature on survey research is 
helpful as a sburce of general prin- 
ciples of interviewing applicable in 
studies of the .mentally retarded. 
In Chapter 2, for example, we dis- 
cujijsed many of the vari^les such as 
qiiestion wording, interview behavior, 
and subject characteristics which 
can produce error in any survey 
research project. Field interviewer 
manuals developed for surveys of the 
general population are also useful. 
For examplSf we have found the 



interview ri " s Manual developed by 
the Survey Research Center at the 
university of Michigan (1969) highly 
informative about. everything from 
sampling to closing the interview 
gracefully. 

On the other hand, we feel that there 
are some unique problems in con- 
ducting survey research with the 
mentally retarded. As later chapters 
will demonstrate, obtaining meaning- 
ful answers from low verbal persons 
is problematic and requires special 
techniques. In this chapter, we 
focus on the logistics of interview-^, 
ing the xetarded, drawing on the 
lessons we have learned through trial 
and error in hopes that others may be 
spared our errors. 

Designing Interview Schedules 
for the Retarded 

In any survey research projeqt, 
the researchers presumably have an 
idea of V7hat they are interested in 
learning and are guided by this ±deu 
in developing questions. The mci.e 
specific the idea is the better; thus 
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it is iitportant as a first step to 
outline in some detail the topic 
areas of interest, the information 
goals. It is also important where 
possible to anticipate which topic 
areas may be difficult for mentally 
retarded persons. Much of our data 
will pinpoint topics about which we 
had great difficulty obtaining in- 
formation. There were other topics 
we tended to avoid from the start 
because we anticipated problems. 
Generally, it has been our ^experi- 
ence in previous interviewing re- 
search that many mentally retarded 
persons have difficulty with time, 
number, and money concepts (e.g., 
giving a chronological account of 
their whereabouts or their time 
allocations durii the day, or in- 
dicating the number of times they 
have done something in the past 
month, or describing their incomes 
and sources of income) . Words 
such as "before," "since," and 
"after" are difficult (e.g., "Where 
did you live before that?") . We 
have also found it difficult to ask 
about experience with service agen- 
cies, as clients often view the 
agency as "Mrs. Brown" or "Mr. Tom's" 
and do not have rnmplete under- 
standing of an aqeuoy and its func- 
tioning. We have also found it dif- 
ficult to ask hypothetical ques- 
tions which ask ! the respondents to 
imagine a different world from the 
one they currently experience. 
Some of these areas are problematic 
enough that, where possible, the 
information is probably best sought 
from nonretarded persons who know 
the subject well. In other cases, 
the process of developing questions 
will simply be more challenging and 
time-consuming • 

Surveys of the mentally retarded 
population will almost inevitably 
involve some branching in the inter- 
•/lew schedule; that is, certain re- 
sponse^' will dictate that succeeding 
,qu nti'. .1 the topic be skipped. 
One must know enough about the kinds 



of persons one is to interview and 
their situations to be able to 
design meaningful subsections and 
anticipate exceptional cases to 
whom questions will not apply. 
However, there is often a risk 
that the distinctions the research- 
ers think they are making will not 
be accorded the status of distinc- 
tions by retarded persons. For 
example, when asked if they work, 
many clients who are in sheltered 
workshops, work activity centers, or 
simply helping out their relatives 
on an informal basis will say "yes," 
The researcher may have sought a 
distinction between being competi- 
tively employed and bel. g unemployed, 
but retarded inter^^lewees may make 
quite a different distinction be- 
tween work and nonwork. Further 
problems arise when clients do not 
distinguish between sheltered work- 
shops, activity centers, and so on. 
In fact, they may know these types 
of settings only by their local 
proper name. Follow-up questions 
are oftan needed to make possible 
the kinds of distinctions that the 
researchers had in mind. 

Similarly, we have encountered many 
difficulties associated with ques- 
tion phrasing, such that respondents 
may not be making the saitie distinc- 
tions we choose to make between, 
for example, being able to do some- 
thing, doing it at least sometimes, 
doing it regularly, or being allowed 
to do it. The more prior knowledge 
one has of retarded persons and 
their view J of the world, the more 
likely it is that questions will 
be sound. However, pilot testing is 
the bes-t single method of checking 
to see that the questions are work- 
ing and determining whether addi- 
tional follow-up questions might be 
needed to clarify answers. 

There is a natural tendency in the 
question do'^^ -'H^ phase for 

questions <,o -'a <ioe, '>>^^ 
naturally f '^\s nus ^^pf ificlty 
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about any area to be covered, but 
also tends to become sidetracked 
in fleshing out an area of question- 
ing. Having clear information 
goals as a guideline is one check 
against this tendency, but later 
editing may still be called for 
after pilor testing.* It is our 
feeling that the interviews them- 
selves should not consume more 
than about half an hour or it be- 
comes difficult to maintain active 
attention. 

Our interviewing studies have each 
required two major forms besides 
a client interview schedule: a 
record -orm^ and a significant other 
interview- record form is used 

to collect whatever information 
is available from agency or facility 
files about demographic character- 
istics, IQ score, length of insti- 
tutionalization, daily schedule, 
and so on. In our studies, parallel 
interviews with "significant others" 
(parents, caretakers, or others who 
know a client well) have' typically 
asked almost all of the same ques- 
tions asked of retarded persons 
directly, excluding some subjective 
questions. This was done in order 
to explicitly examine the corre- 
spondence of answers . In most 
studies, however, there are some 
types of information, as we suggested 
above, that are best gotten from 
significant others or caretakers 
for which the interviewer would at 
least want to seek verification of 
answers given by retarded persons. 
In many studies, significant, others 
might also be used 'as the major 
source of information about such 
things as clients' histories, cur- 
rent skill levels, and degrees of 
independence. Indeed, we find it 
hard to imagine an interviewing 
study which would not require at ^ 
least a minimal amount of data from 
files or from knowledgeable informants 
We found it best to interview such 
informants personally rather than 



have them fill out the questionnaire 
on their own, unless the questions 
were very simple and straightforward. 
The same principles .of survey re- 
search applicable to retarded per- 
sont^ apply to their significant 
others, for their answers too are 
subject to bias, and, in' some cases, 
they lack knowledge about certain, 
areas. This is true even of parents. 

Pilot Testing " . 

Assume that careful planning 
has led to the development of an 
interview schedule. As noted above, 
much of the information in the 
present report is intended to guide 
the task of designing questions 
which can be answered reliably and 
validly by mentally retarded per- 
sons, so we will not discuss ques- 
tion design further here. What 
we want to stress is that what 
appears to be a good interview 
schedule may not turn out tp be one 
in practice. Take the simple ques- 
tion, "Do you make your bed?" As 
our data will indicate, this yes-no 
question may be subject to acquies- 
cence which inflates the number of 
interviewees reporting that they 
make their beds. Moreover, the 
question is ambiguous. It does 
not indicate, for example, where 
bed making takes place. Often in- 
stitutionalized, persons do some 
things at home that they do not 
do in the institution and vice versa. 
The question also leaves open the 
question of extent of activity since 
a "yes" response may mean that the 
respondent has made a bed once or 
that he or she makes a bed every day. 
In constructing questions, one must 
adopt a highly critical attitude, 
actively looking for ambiguities 
and actively imagining potential 
misinterpretations of the question. 
It is helpful to imagine interview- 
ees in diverse situations to judge 
whether a question is indeed appli- 
cable and meaningful to the whole 
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range of persons to be interviewed. 
Instead of thinking about the gen- 
eral rule, one does best thinking 
about all imaginable exceptions to 
the general rule. 

Imagination can carry one only so 
far, however. Pilot testing of 
interview schedules is, in our opin- 
ion, essential. A first step 
may be role playing the interview 
with co-workers playing retarded 
persons. This at least provides 
an estimate of the approximate 
length of the interview and iden- 
tifies questions which may have 
looked good on paper but do not 
soxind good when they are asked. 
Those playing the role of inter- 
viewees can be asked to actively 
search for chances to misinterpret 
the question or raise questions 
about* its meaning. The next step 
should be pilot interviews with 
retarded persons similar to the 
ones who will actually be inter- 
viewed. Ideally, these pilot inter-i 
views should be tape recorded, 
and the interviewer should feel 
free to rephrase questions on the 
spot if they do not seem to be under- 
stood. Such rephrasings and the, 
re^Vonses to them are o^ten useful in 
• refining questions. Th^Txnterviewer 
can also probe to detei/mine if the re- 
sponse given is what wie interviewee 
meant to say in cases^ where the validi 
ty of response seems suspect. We have 
found that conducting pilot interviews 
inevitably reveals problem areas that 
would otherwise go undetected: ques- 
tions that few can answer, questions 
that are greeted by puzzled expres- 
sions, questions that are quite consis- 
tently misinterpreted, and questions 
or questioning s.trategies that prove 
embarrassing for' both interviewer 
and interviewee. In order to gather 
information about the validity of 
responses, it vould also be useful 
to compare clients' answers to 
documented fact or selected responses 
given by nonretarded informants in 
parallel interviews. 



Interviewer Training 

It is our opinion that virtu- 
ally any sensitive and conscientious 
person can become an effective inter- 
viewer of retarded persons. The 
more experience the interviewer has 
had with retarded persons , the 
better, however. We suspect, although 
we did not eticounter the problem 
ourselves, that some people, un- 
familiar with retarded persons , may 
not work out as interviewers if they 
have negative reactions and those 
reactions are noticed by interviewees* 
In previous research, we actually 
encountered similar problems with 
some persons experienced in mental 
retardation — usually agency staff 
who carriec* an authoritarian atti- 
tude into their interviews. Assum- 
ing that willing and able persons 
are recruited, the main tasks are 
to train them to use the interview 
schedule and to standardize their 
behaviors as interviewers. 

Interviewers should, of course, be 
well-versed in the content of the 
interview schedule before they con- 
duct interviews. A serious pilot- 
ing effort can provide the setting 
for training. Where feasible, it 
is useful to have two interviewers 
record the amswers given by pilot 
subjects so that the second inter- 
viewer cam provide feedback on the 
first's approach and the data 
collected by the two can be compared 
for reliability* 

Interviewers not only have to be able 
to use the interview schedule 
confidently, but they must be well- 
versed in all aspects of conducting 
the interview. They should have a 
common understanding of any ques- 
tion rephrasing or probing procedures 
to be used; they should have a 
consistent approach to providing non- 
specific feedback to interviewees 
to maintain rapport and encourage 
answering; and they should establish 
in advance how to handle various 
problems that may arise. Research 
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discussed in Chapter 3 suggests 
that interviewer characteristics 
and behaviors can at times influence 
responses. Certainly if one inter- 
viewer is highly warm, supportive, 
and attentive while another goes 
through the motions in a cold and 
mechanical manner, one can expect 
very different results. There is 
also the risk that a warm and re- 
sponsive interviewer can become 
more mechanical after hours of back- 
to-back interviews. Thus, inter- 
viewer manner can be as important 
as interview content. 

Locating Interviewees 
and Scheduling Interviews 

In some types of studies in 
which s\jbject's addresses have been 
obtained^ from an agency or facility, 
simply legating the subject and 
arranging^ for an interview can be 
problematic. When one arrives at 
the supposed address and finds that 
the client\ no longer lives' there, 
one can check with neighbors, the 
telephone ahd utility companies, the 
Post Officd, or the Police Depart- 
ment. City directories usually on 
file with the Chamber of Commerce 
can also be useful. Other potential 
sources of | information include local 
churches and small neighborhood 
grocery stores. The interviewer 
on a search should always explain 
his or her affiliation and purpose, 
but should avoid indicating that the 
persons being sought are mentally 
retarded. As Edgerton (1967) has 
vividly shown, for example, many 
deinstitutiionalized persons have 
struggled Wo avoid labeling and 
stigma; they would not appr<=»ciate 
having their "covers" blown 
by an insensitive interviewer. 

i 

Initial contact with an interviewee 
can be made either by telephone to 
set up an appointment or by going 
to the home or residential facility. 
Often cooperation is more likely 
if one goes directly to the home. 



but telephone contact is more 
cost-effective and works well if 
the client has already pro\ ..ded 
consent by mail, if telephone 
contact is made to schedule appoint- 
ments, it will still be necessary 
to introduce yourself and the study 
briefly when you arrive for the 
interview as time will have passed 
in the interim. Plenty of time 
for travel and unanticipated de- 
lays (often informal chatting with 
interviewees or their family) should 
be allowed between interviews. 

Obtaining Consent 

In Appendixes B and C, v/e pre- 
sent additional information on re- 
search guidelines relevant to survey 
research as well as sample consent 
and explanation forms used in the 
present study. The points below 
simply summarize the kinds of in- 
formation needed to make consent 
informed consent. There is a special 
burden on interviewers of the re- 
tarded to insure that the study is 
understood, and extra effort and 
patience may be required. The em- 
phasis in ccmmiinications with retarded 
persons as well as their parents 
and care providers should be on 
simple, concrete, and even purposely 
redundant language, in obtaining 
consent for an interview, explain 
the following in simple, concrete 
language : 

1. Who you are 

2. Who the interviewer will be 
(if it is not you) 

3. For whom you are doing this 

4. Why you want to talk to them 

5. How long the interview will 
take 

6. What kinds of questions you 
will be asking 

7. When you will be conducting 
the interview (approximately; 
schedule specifics based on 
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their schedules and desires 
as a matter of courtesy) 

8. Whera it will take place 
(Typically their own living 
situations put them most at 
ease, but it should be left 
open in case they prefer 
another place) 

9. Who else is involved — e.g., 
how you got their names, 
others you will be talking to 
about them, etc. 

10. How it will affect them (or 
the fact that it will not 
affect them if all informa- . 
tion is to be kept confiden- 
tial) 

11. Who will see their answers 
and who will receive any re- 
ports that are based on the 
answers of many interviews 

12. How you will be asking for 
their consent and what they 
need to do to give it 

13. That participation is vol"un- 
tary — that they do not have to 
be interviewed if they do not 
want to, that they can end 
the interview at any time, 
and that they can skip ques- 
tions they do not want i-o 
answer 

14. Whom to contact ai^d how to do 
so should they ha /e ciny ques- 
tions 

Depending on the regulations for 
access to subjects in effect, these 
points are either presented in a 
letter requesting consent to partici- 
pate or they are presented orally 
in person or by telephone, before 
the interview is to be conducted. 
Our preference is for the oral pre- 
sentation not only because it gives 
the client the opportianity to ask 
questions and clarify any misunder- 
standings, but also because the re- 
turn rate- on mailed consent forms 
is usually quite low. Moreover / 



when written consent is obtained 
by mail, there is often a long time 
lag between the giving of consent 
and contact with the consenter 
for purposes of scheduling an inter- 
view. At that time, a rehash of 
the important aspects of the study 
is often requir'ed to remind the 
subject of what was agreed to earlier. 
If not previously obtained by letter, 
written consent should be obtained 
after oral consent is given. After 
consent is obtained, we reinforce 
the following points immediately 
before the interview begins: 

1. The interview is not a test, 
and there are no right or 
wrong answers. 

2. Everything said will be kept 
a secret (assuming that is 
indeed the case) . 

3. The interview will not change 
their lives in any way. We 
hope that it will help other 
people like them someday, but 
we are not going to do anything 
to help interviewees with 

any problems they might have, 
nor will we make them do some- 
thing they don't want to do — 
like move or go to another 
school . 

4. Some of the questions may 
sound silly, but we need to 
ask all of them. (Our sur- 
veys indeed involved blatantly 
siily questions as checks 

on acquiescence, but virtually 
any survey contains some ques- 
tions that make interviewees 
wonder. We use. the term 
"silly" rather than "dumb" or 
"stupid" because its conno- 
tation is more favorable. 
We also point out other pe- 
cularities, e.g.) the fact 
that some questions are asked 
wore than once.) 

5. They do not have to answer 
any questions they don't want 



4.6 

66 



to, and can end the interview 
any time. 

6, if they need to stop, to get 
a drink or go to the restroom, 
they should feel free to say 
so. 

Interviews with retarded persons do 
not technically require parent or 
caretaker consent if the client is 
of age and has not been judged in- 
competent. However, it is a reality 
that parents and others who are re- 
sponsible for retarded persons have 
a stake in what happens to them. 
Whatever consent procedures are used, 
it is important to irake contact with 
such significant others so they 
are assured of what you are doing and 
know that it will not affect the cli- 
ent's life or theirs or involve any 
risk of harm to the client. It is 
almost always necessary to make ap- 
pointihents for interviews directly 
through such persons, as you will be 
in their homes or their residential 
facilities. Due to the difficul- " 
ties that some retarded persons 
have with time concepts, such 
persons are also likely to be more 
av/are of client's schedules than 
clients themselves are. The ma- 
jor point, however, is that the 
position of a person responsible 
for the retarded interviewee should 
be respected despite the fact that 
their formal consent may not be 
required. . When parents and care- 
takers understand the study, they 
are put at ease and are. more likely 
to accept your presence. We have 
encountered the most reluctance 
in some residential facilities 
where staff fear the interviews 
may put the facility in an unfa- 
vorable light. In one* interviewing 
study in which we were involved, 
a whole group of clients at one 
facility had to be dropped from 
the sample because houseparents were 
afraid that the study v/ould "dig up" 
something. * TUiticipating these kinds 
of reactioni."; beforehand, you can be 



ready with answers to concerns that 
might be raised. 

Getting Ready to Interview 

We would not think it worthy of 
mention if it had not happened occa- 
sionally, but one can never do 
enough last-minute checking. One of 
^our interviewers, after numerous 
calls and visits, finally lined up 
an appointment for an interview, only 
to arrive, open her briefcase, and 
find that she did not bring the cor- 
rect interview schedule with her. 
Pencils break, mimeographed forms 
sometimes have missing pages, and 
whatever can go wrong often does. 

'when one is interviewing the retarded 
in their homes or in residential facil 
ities, it is important to try to 
find a quiet place away from others 
in the household. We have repeatedly 
encountered difficulties when parents 
or caretakers attempt to take over 
and answer questions for the clients 
or react to. what they are saying. 
The best way to avoid this is to ex- 
plain the need for privacy and your 
interest in hearing what the client 
has to say, even if heXiiDr she may be 
giving factually incorrect informa- 
tion. Sometimes it is helpful just 
to tell the parent that many clients 
will want or expect their parents to 
answer some of the questions and that 
you need to avoid this if possible. 
Also, explaining ,±hat the same ques- 
tions will be asked of them, if that 
is being done, alleviates their fears 
of the client giving out misinforma- 
tion. If you intend to give a ques- 
tionnaire rather than an interview 
to the significant others, it may be 
given to them to fill out while you 
are interviewing the clients. This 
will keep them occupied so they are 
not so likely to be invo'.ved in the 
client's interviews. When the house- 
hold is particularly no;.sy and 
crowded or otherwise i'vimical to a 
successful interview, we have some- 

, times asked the client if ,he or she 



would mind sitting outside for the 
interview. Although parents must be 
allowed to sit in if they insist, 
they may need to be reminded to let 
the client answer the questions. 
This arrangement is not as desirable 
as total privacy, but it is sometimes 
unavoidable. 

Relating to the Interviewee 

Although obtaining consent to be 
interviewed is obviously important, 
it is perhaps more important to move 
beyond mere consent and attempt to 
make the interview an enjoyable ex- 
perience. An interview should be a 
give and take situation. In other 
words, instead of focusing exclu- 
sively on one's own need to obtain 
information, be sensitive to the in- 
terviewee's needs. We have discovered 
that many mentally retarded persons 
find the experience of having someone 
sit down individually with them and 
listen to them rare and gratifying. 
They may want you to look at their 
pictures, listen to them talk about 
something unrelated to the inter- 
view, or meet their friends. Often 
you can suggest that such things be 
done as soon as the interview is 
concluded, and this will satisfy 
them "as well a§ provide an incen- 
tive for continuing the interview. At 
other times it is best, particular- 
ly when clients want to tell you 
something important, to take a break 
on tiie spot and let them talk. This 
is why it is necessairy to schedule 
more time than is required to do the 
actual interview. If an interviewee 
persistently wants to pull you off 
the course, politely steer him or 
her back to the questions, with 
something like, "That's interesting. 
Now I'd like to ask another ques- 
tion." 

On occasion, a mentally retarded 
person will display inappropriate 
behavior in an interview. Our in- 
terviewers have encountered the ga- 
mut, from masturbation, to pulling 



the hair of the interviewer, to 
getting up and wandering around the 
room, to simply carrying on a mono- 
logue totally unrelated to the in- 
terview. One of our interviewers 
simply ignored the fact that her in- 
terviewee was masturbating during 
the interview, but not all inter- 
viewers will find this quite sound 
approach of ignoring inappropriate 
behavior acceptable. If, as occa- 
sionally happens, a client poses a 
physical danger to the interviewer 
or to himself, terminating the in- 
terview is the obvious course of ac- 
tion. In response to most disrup- 
tive behavior, the best policy is to 
be firm but pdiite in attempting to 
steer the client back to the inter- 
view as well as to reinforce, 
through smiling and words of encour- 
agement, paying. attention and answer- 
ing questions. 

Perhaps the most important word in 
interviewing retarded persons is re- 
spect^. Throughout history, mentally 
retarded persons have been treated 
as less than fully human individuals. 
Repeatedly they find themselves in 
situations where two nonretarded 
people talk about them as though 
they were not there instead of ask- 
ing their opinions' directly. In 
scheduling an interview, in con- 
ducting it. in closing it, the in- 
terviewer should remember that the 
interviewee's needs come first, that 
he or she is helping you. It is not 
enough to think, "I'm going to help 
retarded persons through these in- 
terviews and I'll make them feel 
that their contrjibution is impor- 
tant." There is a major difference 
between this attitude and feeling 
that the interviewee actually is 
important and can help ycu. Re- 
tarded persons can detect this dif-^ 
f erence . and are likely to respond 
much more openly to the extent that 
you give them reason to trust you 
ciid come across as ^a person who is 
genuinely intiEirested in them and 
their opinions, whatever they are. 
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For some mentally retarded persons, 
especially those who have left in- 
stitutions for the. community or 
whose present situations are in jeop- 
ardy/ there may be a threat implied 
by the interview. In particular i 
formerly institutionalized persons 
and their guardians may worry that 
their answers will somehow result in 
their being returned to the institu- 
tion or being forced into other un- 
desirable living, work, or school 
settings. They may perceive the in- 
terviewer as an adult with power over 
their lives. It it therefore essen- 
tial to emphasize strongly both the 
confidentiality of the information 
they will, be providing and the fact 
that being interviewed will not 
change their lives in any way. Of 
course, if the project is such that 
it may affect their lives, they 
should be informed honestly of this 
during the initial explanation pre- 
ceding consent. Hopefully your hon- 
esty with them will help put them at 
ease and encourage honest answers. 

Sensitivity to the interviewee's 
needs during the scheduling and con- 
sent process is a start, but trust 
must also be nurtured throughout the 
interview as well. One way to do 
this is to provide nonspecific posi- 
tive feedback along the way. It is 
dangerous to praise or endorse spe- 
cific responses during the interview 
because you may inadvertently shape 
future responses. To provide nonspe- 
cific feedback, you might say, 
"That's really interesting," or "I'm 
glad you told me that." Smiles and 
nods after responses are even less 
specific and should be used frequent- 
ly. You should generally have an in- 
terested .look on your face and avoid 
any expression of disapproval of what 
is said. It is best to put any ques- 
tions that can be considered sensi- 
tive near the end of the interview in 
hopes that by then a relationship of 
trust will have been fostered. How- 



ever, avoid ending the interview on a 
depressing note, by returning to less 
sensitive questions at the very end. 

Questioning Approaches 

Since many retarded persons have 
both limited verbal skills and vari- 
ous speech defects , simply under- 
standing what they are saying can be 
a challenger and there are undoubted- 
ly differences between interX^iewers 
in the extent to which they can de- 
code difficult-to-understand answers. 
Parents and others who know the cli- 
ent well may be able to understand 
much more and can translate for the 
interviewer, although as we have 
noted, the presence of such persons 
also has definite disadvantages 
because clients may not feel com- 
fortable saying certain things in 
their presence. Also, parents in 
particular may take over the task 
of answering. We still prefer that 
interviews be private, and suggest 
that the interviewer elicit repeti- 
tions of answers when they are r^t 
understood. With more verbal cli- 
ents, it may be enough to ask, 
"What? I didn't hear," but with less 
verbal ones it is often necessary to 
actually reask the question ("Let me 
ask that: again; I couldn't get your 
answer.") Although it is frustrat- 
ing for both the interviewee and in- 
terviewer to go through this process 
repeatedly, patience may pay off. 
As long as it is done politely, most 
clients will try harder to get their 
messages through. 

Let us illustrate the virtue of pa- 
tience. One adult living in the 
community was asked, "What things 
do you not like .about living here?" 
She said an unintelligible word 
twice; then the interviewer asked, 
"The what?", and she said the same . 
unintelligible word twice more. Af- 
ter another round, the interviewee 
finally pointed up to the sky and 
gestured to indicate that she was 
talking about noise from airplanes. 
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(She lived near a busy airport) . 
This woman has a serious speech' de- 
fect; rhus there was a continual 
need to reask questions in order to 
understand what she was trying to 
say and a continual resorting on her 
part to gestures. It was indeed a 
frustrating interview for both par- 
ties and it did not yield much in 
the way of information, but at least 
the interviewer's patience and per- 
sistence gave the respondent the 
fullest opportunity to speak for 
herself. It is certainly preferable 
to go through ,this reasking process 
to obtain clarification rather than 
to jump to^ an interpretation of 
something said which may be com- 
pletely wrong. 

It may be tempting to check out 
your hypotheses by saying, for ex- 
ample, "Did you say you like to go 
swimming?" Unfortunately, since we 
have discovered acquiescence to be 
a problem in interviewing retard'/ 1 
persons, this may not provide veri- 
fication of the response. Some cli- 
ents may say "yes" simply to be 
accomodating. This technique is 
potentially usable with some clients 
if you have reason to believe that 
acquiescence will not be a problem, 
but we strongly favor, reasking the 
question as the first strategy .^^hen 
responses are not intelligible or 
relevant. In our own research we 
generally reasked questions only 
once, but additional reaskings might 
be an even better policy. Then, if 
that fails, a good s'>trategy to use 
when it is necessary to get clarifi- 
cation is to ask an either-or ques- 
tion. This often works even when 
neither choice is correct. They are 
more likely to correct your wrong 
assumption if the probe is struc- 
tured as an either-or question than 
if it has a yes-no structure. 

In view of findings to be presented 
later on the effects of question 
phrasing and wording on the types 
of answers given/ we would also em- 



phasize che need to stick with the 
interview schedule even when cli- 
ents do not appear to be able to 
respond appropriately to a question. 
If interviewers are given the laci- 
tude to rephrase questions if the^ 
original question does not yield 
an answer, they have a natural ten- 
dency, as do adults speaking to 
children, to simplify the question, 
often by converting a difficult 
question format into an easier yes- 
no question. If the interviewer 
feels it is necessary to clarify 
a response, the question can be re- 
phrased, but the alternatives should 
be presented in the same form as in 
the original question. Sample 4.1 
illustrates this teclinique applied to 
a multiple choice question. As our 
data will show, this can substan- 
tially affect the responses that are 
given. Similarly, rewording the 
question, even without changing its 
format, may in effect ch'.nge the 
question. Even a change in emphasis 
on certain words can change re- 
sponses (e.g. "Are you usually by 
yourself?" vs. "Are you usually by 
yourself?"). The first will elicit 
responses based on present feelings 
more than A:he second because the 
word "usually" may not be processed 
unless it is emphasized. In short, 
if cilternative ways of asking the 
qiaestion are carefully planned in 
advance to minimize any potential 
for altering the intent of the orig- 
inal question, rephrasing might 
prove valuable. Generally, however, 
we favor giving exactly the same 
stimulus to each person interviewed, 
and simply treating as missing data 
cases in which an answer to the 
question could not be provided. 

Also, when presenting multiple al- 
ternatives it is important to present 
all of them. Do not take a subject's 
response unless he or she has heard 
or seen all of the alternatives. Of- 
ten, subjects will try to interject 
an acquiescent response after each 

.10 
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Sample 4.1 

Rephrasing a Multiple Choice Question without 
Leading the Respondent 

The following is an example of how a multiple choice question might be re- 
phrased to obtain an appropriate response without changing the form-it or 
intent of the question. The subject here had already indicated that he 
watches TV. 

I: no you watch TV a lot, some, or not much? 
S: Huh? 

I: Do you watch TV a lot, some, or not much? 

S: Most every night ^ 

I: Would you say that's a lot, some, or not much? 

S: When "Thn Hulk" is on. Sometimes I listen to Elvis records. 

I: I need you to pick one of the answers here. (This was said in a 
friendly manner, so the siobject would not feel intimidated.) Do 
you watch TV a lot, some, or not much? (Vocal emphasis was put on 
all three alternatives equally.) 

S: I '11. say sometimes. Yeah, some. 

Contrast the rephrasing techniques used above with those found in the 
following example. Note, how the interviewer leads the subject into a respor 

I: Do you watch TV a lot, some, or not much? 

S: Every night I can. 

I: Is that a lot? 

S: Yeah. 
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alternative is presented. To avoid 
this, do not pause between alterna- 
tive, even if a response is inter- 
jected. You will probably have to 
repeat the question, but by doing 
this on the first asking, you have 
indicated that a yes-no response is 
not appropriate. Most subjects will 
wait for the full presentation on the 
second! asking. Non-verbal cues are 
also beneficial. Repeat the phrase, 
while you are looking at the inter- 
view schedule. When all alterna- 
tives oave been given, then look at 
the client to indicate you want a 
response. 

Similarj logic applies to the issue 
of probing in response to inter- 
viewee responses tc seek additional 
information. In some interview stud- 
ies where in-depth material is 
sought, this kind of probing can be 
invaluable because it allows the in- 
terviewer to build on what the cli- 
ent says to construct a fuller plc-- 
ture. However, we feel that siuch 
probing techniques should also be 
worked out in advance. We would 
again warn against the use of yes-no 
questions as probes, and would favor 
either-or questions and open-ended 
questions in order to minimize the 
chances of leading the respondent in 
a direction he or she may not have 
intended to go. Similarly it is 
dangerous to use reflective tech- 
niques of encouraging interviewees 
to talk more (e.g., "You didn't like 
that houseparent, did you?") unless 
you are completely confident of your 
understanding. Again, less specific 
types of reflective feedback, simply 
saying "uh huh," "oh, I see," and the 
like, are preferable. The transcript 
in Sample 4.2 illustrates an inter- 
viev/er's use of probes to clarify a 
response without leading the respon- 
dent. 

There are often oc.casiors in inter- 
viewing retarded persons when it be- 
comes very difficult to separate fact 
from fancy. Sometimes, what is being 



said simply does not sound plausible* 
For example we interviewed a man who 
claimed that he was married to the 
person assigned as his citizen advo- 
cate and no amount of probing would 
dissuade him from that story. An- 
other adult living in the community 
was asked the question, "Is anyDoay 
teaching you about using money,, 
now?" She replied, "I did have, but 
she was working for the state but 
she charged money and I couldn't 
afford to pay." Later, when she was 
asked if she received help from the 
vocational rehabilitation office/ 
she returned to this theme, but even 
through additional probing it proved 
impossible for the interviewer to 
clearly establish who this person 
might have been, for the closest the 
interviewee came was to describe her 
as a social worker and relate how 
the woman did not like the client's 
barking dog but still "begged" to 
come every week. She finally told 
ithe woman not to come: "I said, 
'Take my money? No thank you.' I 
said I ain't got that much money to 
pay everybody comes to visit me." 
.'Because clients so often do not know 
what agencies people represent or 
/forget their names, it is very dif- 
/ ficult to follow such a story and 
determine if there is any factual 
basis to it, even though the story 
itself is otherwise rich in detail. 
Verification with someone knowledge- 
able about the interviewee woij^ld be 
useful, but this cannot always be 
done. When you have clear e^'idence• 
that what the client is sayin^- is 
wrong, you can eliminate the data. 
On the other hand, one must avoid 
assuming that a client is wrong, 
just because he or she describes 
such things as exploitation or mis- 
tfieatment by service providers, and 
one should be sensitive to the 
feelings underlying even fabricated 
stories. . 
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Sample 4.2 
Exarnple of Probing Techniques 

In one of our studies involving unstructured interviews with adults 
living in the community, the interviewer was free to pursue answers to 
structured questions through probes, but attempte 1 to do so v?.thout lead- 
ing the interviewee, as illustrated in this example, whe^ s probing wa 
essential to establishing whom the interviewee was talkii at. In 

response to a question askinq who her best friend was, the interviewee 
said "Lois" and then cfter three structured questions about the friend, 
the interviewer asked, "How did you meet Lois?" 

S: The lady I used to have was named Donna, and she. I don't know how 

to say it. She got me to know her and that's how. 
I: Now who is Donna? How did you know her? 
S: My counselor. 

I: Was Donna your counselor, or did your counselor introduce you to 
Donna? 

: . She was my counselor and she introduced me to Lois. 
I: O.K. Where does Lois work. 
S: MMMm 

I: Is she just a friend, or does she help you? 

S: She helps. She trying to help me find a job, too. ^ 
I: O.K. Is Lois a counselor, or is she just a friend? 
S: A counselor ^ . . ' 

I: DO you know who she's a counselor for? 

S; It's on ^Road, the one I know, the place on ^Road. 

(With the help i tinwer to a subsequent quc^. i J firmly es- 

tablished that Lois was a rehabilitation counselor.) ^ 
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Closing the Interview 

After the official interview is 
completed, we ask if the respondent 
(1) has anything to add or wants to 
talk about anything else, or (2) has 
any questions he or she would like 
to ask. It seems only fair, since 
they have helped us, to give them a 
chance to use our time as hey wish. 
Sometimes, if interviews h /e delved 
into sensitive or bothersome areas f 
the client may want to continue talk- 
ing about these areas and this may be 
an i:nportant" opportunity to work out 
any unresolved tensions. If you 
know that a client became upset about 
some topic raised in the interview, 
it may be helpful to allude back to 
it afterwards and give the inter- 
viewee a chance to let off more 
steam. This is also often tha time 
to let the client show you something 
or introduce you to someone. 

In our experience, informal chatting 
at the end of the interview often re- 
veals more insights than the inter- 
view itself. Interviewees are more 
relaxed because what: they- say is "off 
the record." In some cases, they may 
express negative ■ ns which they 
were afraid to ex^ while they 

were "on the record." It is, of 
course, unethical to surreptitiously 
tape such comments, but you may ask 
to leave your tape recorder on if 
this information is to be collected. 
Writing do\m what they say puts the 
situation "on the' record" again and 
discourage'i openness. Another possi- 
bility is to take notes after you 
have left, particularly if what is 
said adds a different perspective to 
what was gained in the official in- 
terview. We are not sure what to do 
with such information when it clearly 
contradicts responses given in the 
inter'^l^w. In a r^^ill-scale study, 
one mi<jht actual -^-^^ise answers in 
the interview ^is of infor- 

mation surfacing in post-interview 
discussion, but there are dangers in 
doing so. It may be best to leave 



answers as they were but draw on 
post-interview comments to enrich 
one's understanding of the perspec- 
tives of retarded persons or prepare 
in-depth case studies. 

Our main point is that you as the 
interviewer owe the interviewee 
some of your time if he or she 
wants itr (Sometimes they do not,, 
and the task is one of recognizing 
that even if they are too polite to 
say so.) It is rude and possibly 
disturbing to the interviewee to 
make a hasty exit just as soon as 
you have gotten what you wanted out 
of the relationship. We prefer 
viewing the interviewer- interviewee 
relationship as a human relationship 
and would like to think that the in- 
terviewer leaves* in such a manner 
that he or she would readily be wel- 
comed back. Profuse thanks to every- 
one involved for their time and help 
are obviously, called for. 

Recording Interview Data 

In our research, interviewers 
have attempted to write verbatim 
responses as ' * '^^^ are given. Al- 
though tape rc Ing of interviews 
can be useful, it . two major dis- 
advantages. First, it can be a dis- 
traction or source of anxiety for 
some subjects and may make them less 
likely to be fully honest. This may 
be alleviated somewhat by explaining 
that the tapes will not be heard by 
anyone else, so that you're taping 
the. interview so you caii listen to 
it yourself to make sure you got 
everything that was said. Second, 
transcription of tapes is extremely 
time-consuming and difficult, es- 
pecially when the quality is poor due 
■ to background noise in residential - 
settings or -^"^-^n interviewees 
have speer.. flculties or impedi- 
ments. In studies involving in-depth 
and largely unstructured i iterviews, 
there is probably no alternative to 
taping, but we would still recommend 
that the interviewer, take as many de- 
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tailed notes on the spot as pcyssible 
and use the notes as the primary 
source of data, understanding at the 
time of the interview is typically 
greater than understanding days later 
when listening to a low quality tape, 
partly because mouth movements and 
body lanquage are aids to understand- 
ing. In short, we would recommend- 
use of tapes as back-ups but not as 
primary data sources. We ourselves . 
used tape in this fashion in conduct- 
ing relatively unstructured inter- 
views with highly verbal adults liv- 
ing in the community. 

Otherwise, we have found it feasible 
to write down verbatim responses as 
most retarded persons are not highly 
verbal. If they do launch into ir- 
relevant monologues, these responses 
do not need to be recorded. Writing 
does interfere to an extent with 
making eye contact and maintaining 
rapport, so it is important to bear 
that in mind and attempt to maintain 
eye contact as much as possible. Oc- 
casionally we have encountered an- 
other unusual problem. A few clients 
will stop talking when they see that 
the interviewer has stopped writing 
or has run out of room on the inter- 
view schedule to write- a response. 
This is a minor problem that can be 
prevented by leaving large amounts of 
space after questions likely to eli- 
cit long answers. Nonetheless, it 
illustrates the fact that a host of 
unanticipated variables affect be- 
havior in an interview situation and 
the more that can be anticipated and 
controlled the better. 

Finally, we re;Commend that the inter- 
viewer go over the written notes im- 
mediately after the interview to make 
sure that nothing important was in- 
correctly or uninterpretably written 
down. This is essential sir>ce memory 
fades rapidly. 

Coding and Preparation for Analysis 

To the extent possible, cate- 
gories of response should be planned 



in advance of data collection so 
that interviewers can simply mark 
response codes during the interview 
rather than constructing them on the 
basis of written accounts of re- 
sponses. This is quite feasible if 
the interview is highly structured 
and uses yes-no, true-false, either- 
or, and multiple choice formats. It' 
is also feasible with many open- 
ended questions in areas where soiii^ 
previous research has been done and 
one can anticipate what the likely 
response categories will be or has a 
clear idea of how a universe of re- 
sponses should most logically be 
carved up. Typically, however, 
there are some open-ended questions 
for which categories of response 
cannot easily be preplanned. When 
in doubt, it may even be preferable 
to wait until the responses are col- 
lected rather than to use preplanned 
codes and later find them to be in- 
appropriate. On the other hand, our 
experiences in developing coding 
categories for open-ended questions 
given by mentally retarded persons 
have been frustrating. As noted in 
chapter 5, such responses are di- 
verse and often strange, and the 
category schemes evolved are 
sometimes less, than aesthetically 
pleasing. 

In forming categories of response, • 
one aims for a delicate balance 
between the general and the speci- 
fic. We have typically used all 
responses to formulate categories, 
but in a large study a sample of 
responses would suffice. In any- 
thing, it is better to err on the 
side of too much specificity. Cate- 
gories can always be collapsed later 
if only general categories are of 
interest, but general categories 
cannot as easily be broken down 
again into more specific ones. In- 
evitably studies of this nature 
involve some receding after cate- 
gories have initially been devel- 
oped. 



Intercoder reliability may be a 
problem with some questions. We 
have usually established such reli- 
ability informally by Mia inter- 
viewers, after they have attempted 
to establish a set of coding cate- 
gori es based on s imil arities of re- 
sponse, explicitly draw up common 
conventions they will use in decid- 
ing which code to apply. We have 
also done small-scale intercoder 
reliability checks on two or three 
interviews to insure that two dif- 
ferent coders do indeed code the 
same responses the same way. Much 
depends on the specificity of the 
coding categories. It is straight- 
forward to code a response to a sim- 
ple factual question as "yes" or 
"no," but open-ended questions often 
leave room for subjective interpre- 
tation and some type of reliability 
check should be conducted. 

To facilitate preparation of inter- 
view data for computer analysis, two 
approaches seem particularly attrac- 
tive. In our own. research , we used 
interview schedules which included 
in the right-hand column variables 
and computer column numbers (see 



example in Table 4.1). The interviewer 
can either fill in the code on t-^^'^ 
spot, or, as was the case in 
search, due to the fact that 
of both responsiveness and conLu 
of response were done, fill it in 
later. Several blank columns can be 
left f or ~futxrre-xtse-^nca-_c5n^ 
categories for open-ended questions 
have been developed. The advantage " 
is that a keypuncher can punch di- 
rectly from the interview schedules, 
and the step of transferring re- 
sponses to computer coding sheets 
can be bypassed. 

Alternatively, new technologies in- 
volving optical scanning can be used 
to bypass keypunching and the inevi- 
table errors it entails. Response 
codes can be entered on specially 
printed optical scanner sheets, 
enterpd into the computer, and then 
analyzed in any manner in which data 
entered by card can be analyzed. 
Such sheets can be printed to the 
researcher's specifications. This 
approach is especially useful with 
questionnaire data or tests because 
respondents can supply their own 
responses directly on the scanner 



Table 4.1: Coding System Used On Interview Schedules 



1. ?ire you usually happy or sad? 

Sad = 1 Happy = 2 



2. Are you usually with other people? 

No = 1 Yes = '2 



HS 
HSl 

WOP 
WOPl 



C : / ; 
11 

12 



C : 



13 



R : 



14 



The blanks in the right-hand column permit keypunching directly from the actual interview 
schedules. The respondent answered the first question "sad." The "C" in the right-hand 
column indicates content coding, and the "sad" response is accordingly coded 1. The "R" 
indicates a responsiveness code; the 8 represents a minimally appropriate response. The numbers 
beneath these coding blanks indicate the correct column on the cards used in keypunching. 
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sheets. It would not be feasible to 
pr"'nt an entire interview -schedule 
v.i. uch s- , but this approach 
might reduce the number of errors 
inade in translating data into analyz- 
able form, for the interviewer could 
complete the sheets directly from 
the interview protocol- 

Based on one experience, we would 
recommend against the use of what 
is called free format coding in 
which the keypuncher works directly 
from interview schedules and uses 
a special notation system which does 
not involve assigning data to card 
columns during punching but rather 
doing so later. While this tech- 
nique may make it easy to transfer 
data from interview schedules to 
computer cards, it allows room for 
inany keypunching errors and involves 
a number of technical problems in 
then converting the free format data 
to an analyzable form. Because it 
demands more technical sophistication 
than other approaches and because we 
once experienced numerous problems 
with it, we do not recommend it. 



learned about the process of inter- 
viewing retarded persons. Unlike 
subsequent ch nters, this chapter is 
based on sub^ec^ive impressi^ 
rather than hard data. As we have 
suggested, most of the guidelines 
for interviewing members of the 
general population apply equally 
well to interviewing retarded 
persons. We have attempted to 
supplement those guidelines to 
prepare interviewers for some of 
the special problems they may 
encounter in interacting with 
retarded persons. While we have 
tended ^~1^-.^npha size problems, we 
should point ^t^^that the majority 
of persons we have^^^rttorviewed have 
been eager to participate>-e3ctremely 
cooperative, and very gratif iea^^-b^:,^^^ 
the experience of having someone 
seriously listen to what they have 
to say. Similarly, most parents and 
houseparents have been cooperative, 
often because they hope that research 
findings will result in improvements 
in the service delivery system. 



Conclusion 

In this chapter, we have at- 
tempted to share some lessons we have 
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Chapter 5 



TO WHAT EXTENT CAN MENT LLY RETARDED 
PERSONS RESPOND TO INTERVIEW QIjjl.o.^<> S? 



An interview is not a very fruit- 
ful information-gathering technique 
if interviewees cannot ' answer ques- 
tions. In view of the limited verbal 
skills, of mentally retarded persons, 
we decided that the first challenge 
in interviewing the retarded is to 
obtain a response of some kind. Thus 
we measured and analyzed the respon- 
siveness of interviewees, their 
ability to meet the demands of a 
particular question with an answer 
suited to the form of the question 
(e.g., "yes" -or "no" to a yes-no 
question) and relevant to the content 
of the question (e,g., about work if 
the question is, about work) . We. were 
guided by the following questions: 

1. What can be expected of men- 
tally retarded persons in 
interviews / and what types of 
respondent behaviors 'consti- 
tute problems in interviewing 
the retarded? 

2.. To what extent is responsive- 
ness to an interview a predict- 
able individual behavior, and 
what factors are related to it? 



3. How can responsiveness be 
optimized? 

To convey what we have discovered 
about responsiveness, we will take up 
the following topics in turn, drawing 
from the various studies to discuss 
each topic: specific types of 
responses, the stability of respon- 
siveness, the relationship of respon- 
siveness to IQ and other client 
characteristics, the relationship 
of responsiveness to. question type, 
and the optimization of responsive- 
ness. 

Specific Types of Responses 

In most of our -analysis, we dealt 
with-a measure ^of the percentage of 
questions asked of a respondent which 
were answered at least minimally 
appropriately; that is, which met ■ 
the formal and substantive^clemands -.^ 
of the question* However, ^s indi- 
cated in Chapter 3, we also "measured 
various types of inappropriate or 
uncodable responses, and were inter- 
ested in" the extent to which each 
might pose' problems in an interview. 
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Table 5,1 presents the mean percent- 
ages of responses which fell in 
each of the nine responsiveness codes 
in the two institutional samples. 
In each ca^:.- they are based on 
responses to one of the two paral lei 
interview schedule-^ used in these 
studies (82 questions for the chil- 
dren's study and 142 questions for 
t^e adult study) . \ Prof our ;ly 
re carded adults weie excluded becr* se 
the vast majority of them failec' 
the screening inter^);iew and wei 
thereby defined as t^otally unreb. 



sive. Their exclusion makes the* 
child and adult samples comparable 
in IQ (M=42.08 for children, 39,76 
for adults). The means in Table 5,1 
cannot always be taken as representa- 
tive of typical performance because 
some distributions were unusual (for 
example, same clients never gave a 
response while some others nev^r 
failed to respond in some way) • 
However, they are suggestive of the 
freque ^cy with which various types 
of interview behavior can be expected 
^ occur. 



Table 5,1 



Mean Percents of Responses Falling in Each Category 
in Institution Samples 



• ..Responsiveness Code 


Adult . 
Institution 
(N=42)* 


Child 
Institution 
(N=52) 




1, 


No response 


10 . 20 


6,55 




2 . Unintelligible 


' 4,71 


4,91 




3,- 


Irrelevant 


3,48 


5., 11 




4, 


Don't know 


.28 


2,85 




5, 


Inadequate 


; '3,73 


:7,4i 




6, 


Request for more information 


1,01 


3,32 




1, 


Refusal to answer 


,01 


^33 




8, 


Minimally appropriate 


63.63 


57.94 




9, 


Expanded appropriate 


12,99- 


11,58 






100,00% 


100,00% 





*E:?(cludes those with IQs below 20, who vi^re generally totally unresponsive, to make 
groups more comparable in IQ. 
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We can interpret these figures either 
of two ways. We can say, for example r 
that the typical child interviewee 
was able to give a minimally appro- 
priate an^ower to 63.63% of the ques- 
tions on the interview schedule. 
Alternatively, we can say that of 
all the behaviors elicited from the 
sample by interviewers'' questions, 
63.63% of them were minimally appro-' 
priate answers. Clearly in both 
samples we obtained minimally appro- 
priate or expanded answers in a 
majority of cases, more often minimal 
than elaborated. 

What of fhe inappropriate types of 
respondent behavior? Generally, a 
high percentage of inappropriate • , 
responses took the form of no 
response at all. Unintelligible 
responses were also a problem, 
unintelligibility , oi course, being 
a function, of the interviewer's or 
listener's capabilities as well as 
of the interviewee's coinmunica1;ion 
skills. It is highly likely that a 
person who knew the respondents might, 
well have been able to decipher many 
responses that our interviewers had 
'to judge unintelligible, but in m.>st 
surveys interviewer and interviei'ee 
are not familiar with oi^ie another. 
Irrelevant responses, that is, 
answers that appeared unrelated to 
the substantive demands of the ques- 
tion, were a frequent problem as well, 
often because interviewees did not 
understand a particular question. but 
in some cases because interviewees 
.hc.d a general lack of comprehension 
cf the interview but chose o speak. 
Somewhat more common still were 
inadequate responses, which were 
iudged to be relevant to the question 
bat too vague or ambivalent to be 
coded into content categories. At 
times respondents indicated that 
they did not know the answer , but 
this was fairly rare. Our coding 
system was based on an assumption / 
usually but not always v." lid, that 
respondents should have known how 



to answer the questions we asked. 
Similarly, iriterviewees occasionally 
requested more information about 
the question before answering, 
usually by saying "huh?" to elicit 
a repetition of the c^ .ostion, but 
this did not happen often either. 
By far the rarest event was a'" refusal 
to answer the question, even though 
participants had been told at the 
start that they did not have to 
answer any question they did not 
want to answer. 

The relative frequencies of the nine 
typ'es of responsiveness codes were 

• much the same in the two institu- 
tional samples as well as in other 
samples. We would have to conclude 
that interviewers of the retarded 
can expect to encounter a wide range 
of inappropriate responses^ but that 
appropriate responses can generally 
be expected a majority of the time. 

The Stability 

• of Responsiveness Scores 

In both of the institution samples, 
similar interview schedules were 
administered a week apart, allowing 
us to examine the extent to which 
responsiveness is a consistent indi- 
vidual behavior. 'In the children's 
institution sample, For:n A consisted 
of 82 questions and Form B of 65 
questions, with 31 questions repeated 
' in exactly the same manner on both 
administrations and the remainder of 
the questions typically representing 
alternative ways of seeking the same 
information. A total responsiveness 
score on each form was calculated 
for each subject (the number of 
questions answered appropriately 
divided by numbed of questions asked) 
When responsiveness to Form A and 
responsiveness to Form B were cor- 
related, the result was a highly 
significant correlation of .96, ^indi- 
cating that the extent to which an 
interviewee was responsive to one 
form could be predicted almost per- 



fectly from his or her responsive-' 
ness to tlie other form. Although the 
correlations were lower, the frequenr 
cies of specific types of response on 
Form A and B were also significantly 
related, with the exception of the 
category "Request for More Informa- 
*tion." These correlations were .54 
for no response, .74 for unintelli- 
gible, .39 for irrelevant, .30 for 
don't know, .33 for inadequate, .20 
for request for more information, 
.92 for minimally appropriate, and 
.54 for -expanded appropriate re- 
sponses. Thus, the extent to , which 
various less-than-appropriate and 
appropriate responses occurred was 
somewhat predictable from time to 
time . 

In the adult institution sample, 
this finding of stability was con- 
firmed. Indeed, the correlation be- 
tween total responsiveness scores 
for the two forms (Form A, 142 items. 
Form B, 131 items) was identical 
(.96). 

As another way" of examining the sta- 
bility of responsiveness scores, we 
analyzed the extent to which the 
group's level of responsiveness 



changed from first to'^'econd 
administration^ ^In the children's in- 
stitution ,satnple, this analysis focuses 
on 31 -questions repeated exactly the 
same on both -administrations. A Latin 
square design was used tQ exair.-. a the 
administration effect in the./coatext . 
of interviewer effects (see Table 5.2). 
Overall, there was not a significant ^ 
difference between first and second 
administration responsiveness, 
although scores did increase somewhat 
from firs't to second administration 
(from 71.8% to 74.6% overall). There 
was also no oveirall, interviewer 
effect, that is, neither interviewer 
elicited higher rates of responsive- 
ness from the sample. There was 
however, ar^^ interaction between 
interviewer and administration , 
F (1, 49)/ = 9.92, p = .005. Inter- 
viewer Two, for some reason, was 
associated with a greater increase 
in responsiveness frcxn first to 
second administration than was Inter- 
viewer One. It is possible that the 
26 subjects whom she interviewed on 
second administration were, for 
reaspns we cannot explain, more sus- 
ceptible to practice effects than 
were the 26 subjects who had Inter- 



Table 5.2: Responsiveness of Children As a Function of Administration 
and Intervriewer* 



Administration One 



Administration Two 



Interviewer 



One 


Two 


72.20 
73.92 


71.46 
75.28 



*RDr an N of 52 in severe to mild ranges. One half of the subjects had Interviev^ 
One first and half had Interviewer TVjd first, and then ihe other interviev^ oonducted 
ti^ second administration. 
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viewer One on second administration, 
or it is possible that Interviewer 
Two became more relaxed, warm,' or . 
stimulating in manner in the second 
round of . interviews. 

A similar analysis for the adult 
institution sample (see Table 5.3) 
revealed no significant differences 
in responsiveness associated with 
-administration, interviewer/ or the 
interaction of the two. Again^ how- 
ever, responsiveness did increase 
slight J y from first to second adminis- 
tration, and this effect was more 
noticeable for Interviewer Two. 

To put these lines of evidence to- 
gether, then, we would conclude that 
responsivei;iess to interview questions 
is a stable individual behavior. 
Although some respondents may increase 
in -'their ability to respond to ques- 
tions, as a result of practice in a 
prior interview, dramatic changes 
are not to be expected. Moreover, 
an individual's ranking in the group 
as more or less responsive appears 
to be highly consistent from inter- 
view to interview. 



Responsiveness and IQ 

It is auite clear from the liter- 
ature on c^ru^ge development among 
the mer.t:a; *arded (see chapter 2) 

that in the 1 er ranges of the IQ 
continuiim, iuore, delay in language 
and speech is to be expected, and 
more totally nonverbal individuals 
are encotintered. critical question 
in our study of the feasibility of 
interviewing mentally retarded per- 
sons centered on clarifying what can 
be expected at different levels of 
"retardation. "At what point in the 
IQ continuum does obtaining answers 
to questions become infeasible? 

We set out with low expectations 
for persons in the profound range 
of retardation, but nonetheless 
attempted to interview persons with 
lOs below 20 in the study of insti- 
tutioiial.-^ >::ed adults. It was here 
that our simple screening procedure, 
described in Chapter 4, was heavily 
used to avoid the unpleasant exper- 
ience of holding an interview when 
it was apparent' that the subject 
could not participate. As noted, 
the rule was to. accept any verbal 
or nonverbal response, whether intel- 



lable 5.3: Responsiveness of Adults as a Function of Administration 
and Interviewer* 





Interviewer 




One 


Two 




Administration One 


66.15% 


63.37% 




Administration Two 


67.82% 


68.23% 


1 



*For 42 Interviewees In severe to mild range. One half of the subjects had 
Interviewer One first and half had interviewer Two first, and then the other 
Interviewer conducted the second adrr.Inlstratlon. 



5.5 



82 



ligible or not, as sufficient grounds 
for attempting the full interview/ 
Thus, the screening criteria were 
designed to give anyone who showed 
the least sign of ability an oppor- 
tunity to be interviewed. 

Of the 16 profoundly retarded adults 
who were approached, 13 or (81%) 
failed the screening and were judged 
totally incapable of participating 
in an interview. The responsiveness 
of the three vtho did pass the 
screening was jdef icient. Two of 
them were able to respond adequately 
to slightly over half of the ques- 
tions, primarily yes-no questions. 
The third subject failed to give an 
answer two-thirds of the time and 
otherwise gave a high proportion of 
irrelevant answers. 

On the whole, then, our expectations 
about the inf easibility of inter- 
viewing profoundly retarded persons 
were borne out. While three such 
persons were able to respond to some 
extent, we are not confident that 
their answers were meaningful and 
they were the exception rather than 
the rule. Any attempt to seek infor- 
mation from such persons should rely 
on other methods of data gathering; 
systematic behavioral observation 
of circumstances, activity patterns 
and expressed affect, and, where 
feasible, use of manual and other 
nonverbal communication systems. 

All three of the major samples in the 
study (institution children, insti- 
tution adults, and community chil- 
dren) included persons in the severe 
td mild ranges of retardation. Thus', 
we were able to examine in each 
. sample the relationship between IQ 
and ability to respond to interview 
questions for persons with IQs 
ranging from 20 to 68.,-- — 

In tha institution -Children's sample, 
the ability to respond was quite 
clearly a function of IQ. Table 5.4 
represents the mean percentages of 



responses falling in each of the 
responsiveness codes (with the excep- 
tion of refusals to respond, which 
were so rare that they are not 
reported) . It can be noted that 
instances of no response, unintel- 
ligible responses, and irrelevant 
responses decreased as IQ increased, 
while the percentage of appropriate 
responses, both minimal and expanded, 
increa;33d. In sev*^al respects, 
severely retarded subjects, as a 
groups were quite different from 
moderately, and mildly retarded ones. 
Most notably, they were much more 
likely to fail to respond and much 
less likely to provide appropriate 
responses. Overall, the percentages 
of appropriate response to Form A 
for severely, moderately and mildly 
retarded groups were 52.9%, 89.6%, 
and 93.3% respectively, a highly 
significant IQ c;roup effect, F (2.49)=' 
p = .OOOL. 

However, it is important to note that 
variability was also greater in the 
severely retarded group (SD = 38,8) 
than in the moderately retarded 
(SD =9,1) and mildly retarded 
(SD = 3,7) groups. , The respon- 
siveness of individuals in the severe 
retardation range was, in short, 
relatively unpredictable. Four 
severely retarded children failed^ 
the screening, while the most respon- 
sive 4^ve appropriate answers 96„3% 
of the time, and the rest spanned 
the range in between. By contrast. 
Form A responsiveness scores in the 
moderately Retarded group ranged from 
72.2% ta 100%, and in the mildly 
retarded group from;' 83,7% to 97.5%, 
suggesting that all children in these 
IQ ranges were able to answer most 
of the questions they were asked. 
What this suggests is that one cannot 
make firm predictions about whether 
or hot interviewing is feasible with 
severely retarded persons. The only 
way to tell is to attempt an inter- 
view and see how., individuals respond. 



/ 



Table 5,4: Responsiveness Codes by 10 Group: 
Mean Percents of Answers on Form A 



No re'sponse 



Unintelligible 



Irrele^^ant 



Don't khow 



Inadequate 



Request ittore information 



Minimally appropriate 



Expanded appropriate 



Severe 
(N=20) 



25,8 
8,9 
8,0 

0,1/ 

■ / 
3,; 3 

1,1 

50,0 

2,9 

100,1 



Moderate 
(1^=16) 



/ 



0,9 
4.0 
1,1 
0.2 
3.7 
0.6 
72.7 
16.9 
10.0 . 1 



Mild 
(N=16) 



0,0 
,2 
,2 
,6 
4,3 
1,3 
71,6 
21,7 
99,9 



For this sample the overall correla- 
tion bet-ween responsiveness of Form A 
and IQ was .62, and between respon- 
siveness^ of Form B and IQ ,76, again 
indicating a, strong relationship 
between IQ and responsiveness. At 
the same time, there was a negative 
relationship? between IQ and the^ 
percentage of "no response" codes 
(r = .59 f or Vorm A, -,52 for Form B) 
as well as bei^^een IQ and the percent- 
age of all ^oth^^ inappropriate 
responses combined (r = ,36 for Form 
A, -.44 . for FoxttiXb). 

In the adult insti}^uti^|^^ sample, much, 
the same pattern of. f ina^-ngs emerged. ^ 
Overall, of Form A, Revere ly retarded 
persons averaged 58,8% responsiveness, 
and were less responsive than moder- 
ately (78,0%) and mildly retarded 
(•73.5%) groups, -F (2, 39 ='3.63, 
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£ = ,04. On Form B, the same pattern 
emerged ( severe , 56.3%, moderate 76.6%, 
and mild 75.5 %i F (2, 39) = 4.46, 
p = .02. In this sample, one severely 
retarded person failed the initial 
screening. In addition there was one 
mildly retarded subject whose emo- 
tional disturbance seriously inter- 
fered with participation in the 
interviews and whose scores lowered 

the mean for -the mildly-retarded 

group and increased the extent of 
variability. .This woman was not able 
to respond to 90.3% of the questions 
on Form A and 85.3% of those on Form 
B. Her ;exclusion from the^' analysis 
increased the mean of '^Form A for the 
mildly retarded group from 73.5% to 
81.1% and decreased the standard 
deviation from 25.5% to 10.1%.' As. 
in the children's institution study, 
variability in the severe retardation 
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range was high (25,4% compared to 
11.7% in the mod'Brate range and 10.1% 
in the mild range with the outlier 
excluded. ) 

In correlational analysis, the rela- 
tionship between IQ and percentage 
of appropriate responses on Form A 
was a significant but relatively low 
. 35 with the one outlier in the 
mildly retarded group included (and 
.51 with her score excluded). As 
expected, given the high relationship 
between responsiveness scores for 
the two forms, results for Form B 
were similar (.4? with her included, 
.57 with her excluded). Again, then, 
the study of institutionalized adults 
indicated that the ability to respond 
to questions is a function of IQ, 
witti responsiveness dropping off 
markedly in the severe range of 
retardation, but with severely 
retarded persons differing widely 
from each othe"*^- some being incapable 
of participating at all and others 
responding much like moderately and 
mildly retarded persons. 

Our third opportunity to examine the 
relationship between IQ and respon- 
siveness was in the children's com- 
ir.ixnity sample. The interview sched- 
ule used in this study was, in our 
opinion, easier than those used 
previously because we had by then 
analyzed responses to the questions 
used in -the institution samples and 
attempted to refine techniques to 
optimize responsiveness. In this 
sample, responsiveness was indeed 
higher. Severely retarded persons 
did relatively well, averaging 78^6% 
responsiveness, compared to 82.1% in 
the moderately retarded group- and 
92.7% in the mildly retarded group, 
F (2,54) = 5.93, P= .005. In this 
case, both the severely and moder- 
ately retarded groups' were found to 
be significantly less responsive than 
the mi].dly retarded group. The over- 
all correlation between responsiveness 
and IQ was .41. Moreover , .i.n^ this 



sample, the extent of variability 
among severely retarded persons was 
not higher than that among the two 
higher IQ groups (SD =■ 12.8 for 
severe, 18.4 for moderate, and 5.0 
for mild). In our opinion, by 
reducing the number of open-ended 
and multiple choice questions and 
otherwise working to simplify the 
types of questions that were asked, 
we may have succeeded in making the 
task simpler for the community sample 
and reduced the extent to which 
severely retarded persons appear to 
be deficient compared to their higher 
IQ peers. For this reason, these 
figures do not provide a basis for 
concluding that severely ^retarded 
persons in the community are more 
verbal* y skilled than those in an 
institution. 

However, we were able to compare the 
three samples more directly by exam- 
ining their responsiveness to 22 ques- 
tions asked in exactly the same way 
in all three samples. These questions 
included seven verbal yes-no questions, 
nine yes-no questions about chores 
accompanied by pictures, two either-or 
questions, one multiple choice ques- 
tion, and three open-ended questions. 

Vable 5.5 presents the average percent- 
age of questions answered appropri- 
ately in each of the three samples 
at each of the three different levels 
of mental retaidation. A?. _ 
suggests, the f^.ata for tiie different 
samples were somewhat different, and 
chis was reflected in a significant in- 
-teraction effect in a 3x3 analysis of 
variance between sample and level of 
re'i^ardation, F (4, 140) = 4.60^;p = 
.002. This finding required that 
look more carefully, at IQ group differ- 
ences within each sample. In the chil- 
dren's institution sample, the IQ group 
effect v;as sl-j.^if icant F_ (2, '49) = 
14. 48,p < .001, and t-tests indicated 
that severely retarded persons were 
leias responsive' on the average than , 
both moderately and mildly retarded 



Table 5.5: Responsiveness by IQ Group for 2?, Questions 
Used in Three Samples 





Level of Retardation 






Severe 


Moderate 


Mild 




Institu :ion children 


55.17 




94.11 


93.35 




Community children 


84.59 




85.56 


95.68 




Institution adults 


71.59 




90.91 


78.41 





persons. Ih the children's comraunity 
sample, IQ again mattered, F_(2, 52) = 
3.75, p = .03. Here; however the 
severely retarded group was signifi- 
cantly less responsive than the mildly 
retarded group and the moderately 
retarded group was almost so (? = 
.065). In the adult institution 
group, by contrast, IQ group differ- 
ences overall were not significant, 
F (2,39) ^ 2.54,p_ = .09, but follow- 
up t-tests did indicate that severely 
retarded clients were significantly 
less responsive than moderately 
retarded clients (t = 2.87, p = .009). 
In this sample, primarily because of 
the almost total unresponsiveness of 
one emotionally disturbed, mildly 
' reta:rded wdmanv the mildly retarded ~ 
group did' hot have the highest aver- 
age responsiveness. Thus IQ group 
differences were weakest in this . 
group and strongest in the institu- 
tion Ci.ildren's sample. 

The one consistent conclusion to 
be reached is that severely retarded 
interviewees generally can b6 counted 
on to be less responsive than their 
higher IQ peers, but the specific • 
magnitude and locus of IQ group 
difference appear to vary from 
sample to sample. Over all samples, 
of course, the effect of IQ group 



was .significant, F (2, 146) = 15.55, 
p = .^001, with severely retarded 
persons averaging 62.99% responsive- 
ness, as compared to 72.3% for the 
moderately retarded and 77.72% for the 
mildly retarded. 

What of the comparison between 
samples? Overall means for institu- 
tion children, community children, 
and institution adults were 77.45%, 
88.92%, and 79.33%, respectively, 
and differences between samples 
approached statistical significance, 
F (2, 140) = 2.63, P = .08. However, 
since mean IQs for the three samples 
were different, analysis of covar- 
iance was performed to examine 
di-fferences between samples with IQ 
controlled. In this analysis, the 
•samples were not significantly dif- 
ferent in their responsiveness, 
F (2, 145) = 1.16, P = .32) In 
other words, the adult sample had 
no clear adv.antage over samples of 
children, and, more importantly, 
• the -appareht advantage of communi- 
ty children over institutionalized 
persons vanished when^/IQ was con- 
trolled. Thus, contrary to some 
""evidence in. the literature, institu- 
tionalized -persons were not found 
to be notably deficient compared to 
communifc:' residenuS in communication 
skills required to answer questions. 
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Relationships of Responsiveness 
to Other Personal Characteristics 

Although we consistently fonnd 
that responsiveness was related to 
IQ, we found no consistent evidence 
that responsiveness was a function 
of other client characteristics. It 
is most useful to focus on the 22 
questions analyzed above which were 
repeated in all three samples, 
correlations bv^tween sex and percent 
of the 22 questions answered ade- 
quately in the three samples ranged 
from -.02 to .11, suggesting no rela- 
tionship between sex and responsive- 
ness.. Similarly, correlations 
between age and responsiveness were 
weak (-.09 to .12). ,pf course, 
because the age range in the two 
children's samples was restricted 
to 12-16, there was not much oppor- 
tunity for a relationship to mani- 
fest itself in these groups. However, 
the correlation of -.09 in the adult 
samples where ages ranged more widely 
(SD = 4.41) was not significant 
either. As we noted above, there 
were no recognizable differences 
between institutionalized and nonin- 
stitutionalizeu groups either, con- 
trary to some evidence in the liter- 
ature of language deficits among the 
institutionalized. In short, our 
search for other subject character- 
istics predictive of the ability to 
respond in an interview situation - 
was" not productive. However, we did 
find relationships between respon- 
siveness and the type of question 
asked, an important set of findings 
to which we turn next. 

Responsiveness and Question Type 

To illustrate in some detail the 
technique used to ana''*'ze differences 
in responsiveness as a function of 
question type, we will first examine 
the relationships existing within the 
institutionalized children's sample 
and^then explore t^e extent to which 
it was^ similar in the'other sample. 



RESPONSIVENESS AMONG INSTITUTION 
CHILDREN AS A FUNCTION 

OF QUESTION TYPE 

While most of the children's 
institution interview consisted of 
simple yes-np questions, there were 
other types of questions used which 
permitted a comparison of severely, 
moderately, and mildly retarded chil- 
dren's ability to respond appropri- 
ately tu questions as a function of 
question format. Six scales were 
formed, each measuring the percentage 
of questions of a given format that 
were answered with at least a mini- 
mally appropriate response. The 
subsets of items used to calculate 
these scores were a?: follovjs: 

1. Factual Yes-No Questions. Ten 
questions about activities (D 
you: ... read books, magazines, 
or newspapers; go out to the 
movies; go out to eat; go to 
church; go to stores; watcU TV; 
listen to the radio or record 
player; play any sports; go to 
school or take classes; do any 
arts and crafts like ceramics 
or painting?) 

2. Subjective Yes-No Questions, 
Eight yes-no questions, consist 
ing of foiir pairs on the same 2 
topiqs, included as part of an 
attempt to study acquiescent 
response set (Are you usually 
happy? Are you usually sad? 

Do people here treat ^^^^^ nice?" 
Do people here treat you mean? 
Is this place where you live 
big enough? Is this place 
where you live too little? Are 
you usually by yourself? Are 
you isually 'With other people?) 

3. Either-Or Questi-ons . Eight 
either— or questions covering 
the same topics as in scale 

2 above; four pairs with dif- 
ferent orderings of term^-. 
(e.g.. Are you usually happy 
or sad? Are you usually sad 
or happy?) . 
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4.. Verbal Multiple-Choice Ques- 
tions ^ Three questions, each 
supplying four answers from 
which to select (How often 
doefv anybody in your family 
come to see you? A lot, some- 
times, not much, or never? 
How many friends do ycu have? 
A lot, some, not many, or 
none?) 

5. Pictorial Multiple-Choice 
Questions- Three questions, 
each of which asked the 
respondent to select a happy 
face, slightly happy face, 
slightly unhappy face, or 
unhappy face as descriptive 

of his or her feelings (Which \ 
picture shows how you feel 
about living here? Which 
picture sliows how you like 
the food Y^re? Which picture 
shows how( you like the people 
here?) \j 

6. Open-EndBd Questions. Five 
open-ended items (If you had 
one wish, what would you wish 
for? Who helps yOu the most 
when you need help? .What are 
they teaching you there; (at 
school)? Where do you get 
your'money? What do you and 
your friends usually do to- 
gether?) 

With the exception of the yes-no 
iteras^ which_ were_sample^^^ 
of these scales included all items 
on the two forms of the questionnaire 
fitting each format. Our hypothesis 
was that yes-no questions and picto- 
rial multiple-choice questions would 
be associated with the highest ' 
responsiveness rates, and that the^ 
other questions with closed' formats 
(either-or and verbal multiple-choice) 
would elicit higher rates of response 
than would open-ended questions which 
require that respondents frame the 
response. 



Figure 5.1 shows the mean percentages 
of appropriate responses on each type 
of question for each IQ group. This 
analysis by question, subset indicated, 
first of all, that there were signif- 
icant differences among responsive- 
ness rates in the total sample as a 
function of question type (F (5, 45)= 
23.90, p = .0001). For the 50 subjects 
who. had subscale scores for- each of 
the six formats (two' visually impaired 
persons could not complete the picto- 
rial multiple-choice section) , this 
analysis of variance by question ty^e 
was followed up with t^-tests |or_^ 
dependent measures to locate ^^§0if ic 
differences among question types (see 
Table 5.6). Factual ycs-no questions 
and pictorial multiple-choice ques- 
tions were found to be significantly 
easier to answer than all other types 
of questions and were not signifi- 
cantly different from one another. 
The subjective yes-no questions were 
significantly more difficult than 
factual yes-no questions (t^ (49) = 
3.41, p = .001) and bordered on being 
more difficult than pictorial multi- 
ple choice questions as well (t^ (49) = 
1.95, p = .057) . Either-or questions , 
ranked next in difficulty, being 
associated with significantly higher 
lesponsiveness rates than verbal 
multiple-choice and open-ended ques- 
tions, correlations among the six 
subscales (see Table* 5.7) indicated 
that, generally, children's levels 
"""of "nesponsiveneHS— on~~onB- type of 
question were predictive of their 
responsiveness on other types. All 
of these correlations were signifi- 
cant well beyond the .05 level. 
However, correlations among simply 
structured questions were generally 
of a larger magnitude than were 
/ correlations involving open-ended 
questions or the: four-choice verbal 
multiple-choice questions. 

The analysis points out the advantages 
of using yes-no /and pictorial multi- 
ple choice questions when the goal 
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Figure 5-1 Responsiveness to Various Question Types as a Function of Level 
ofRet-ardatlon, 



is to obtain responses from as many 
mentally retarded interviewees as 
possible. As the difference- between 
factual and subjective, yes-no ques- 
tions indicates, the content of the 
question has some " bearing on the 
likelihood of obtaining a response-. 



but is relatively minor compared to 
the effect of question format. Sub- 
jective yes-no questions, and perhaps 
the subjective either-or que ons 
used in this study.^^s well, uluen 
forced respondents ^to make a black- 
and-white choice. Wnere responsive- 
• ■ / 
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Table 5.6: T-Values for Differences between Appropriateness of Response 
Scores on Six Subsets of Items (N=50) 



Pictorial multiple- choice 



Factual yes-no 
Subjective yes-no 



Either-cr^ 



Verbal multiple- choice 



Pict. 



Fact Y-N 
.35 



Subj._Y-N^ 
1.95 
3.41* 



E-Or 



4.27* 
4.28* 
2.38* 



Verbal 
6.14* 
6.43* 
5.20* 
4.17* 



Open 
6.37* 
6.30* 
4.97* 
4.15* 
.25 



♦Significant at the .05 level. 

ness fell off, it was often because easy to answer, undoubtedly because 

residents would request clariiiication they required only pointing and not 

of the question or give vacillating necessarily imderstanding, verbal 

responses which could not be coded mult.' e:-c;ioice questions were, sur- 

one way or the other. VJhile picto- prUinqly, as difficult as open-ended 

rial multiple-choice questions were qnestions. mile this may have been 



Table 5.7: Correlations 7\iivor.g Appropriotc.rs^ss of Response 
Scores on Six Type*^ oE Questions 



Pictorial multiple-cho-i ce 



Factual yes-no 
Subjective yes-no 



Either-or 



Verbal multiple-choice 




♦Significant at the .05 level, 
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partially due to the topics and 
quantitative response options for 
the questions used in the study, it 
as quite likely that, four options 
are too many for many mentally 
retarded persons to consider at once. 
Certainly the two-choice either-or 
questions were much easier by compar- 
ison, despite the fact that they 



to a lesser extent either-or questions, 
raise serious questions about whether 
those formats should be used at all 
with severely retarded children, for 
in each case, less than half of the 
severely retarded group studied was 
able to respond appropriately. More- 
over, it was on these types of ques- 
tions that the difference between the 



RESPONSIVENESS AND QUESTION TYPE 



were subjective and relatively abstract, severely retarded and the high xQ 

^ ^ ^.'^ . groups appeared to be most pronounced. 

Analysis of IQ differences in ^ 

responsiveness to each of the six 

types of questions was conducted 

through one-way analysis of variance 

with follow-up ^ tests. Separate 

variance estimates were used due to 

higher variances in the severely 
retarded group than in the moderately 
and mildly retarded groups. On all 
six sets of questions, IQ group dif- 
ferences, as indicated by F statictics, 
were significant beyond the .005 level. 
Moreover, follow-up t_-tests indicated 
that for all six question formats, 
the severely retarded group was less 
responsive than the. moderately and 
mildly retarded groups and the latter 
two groups did not differ significantly 
from ea(;h other. The only difference 
between the moderately, and mildly 
retarded subjects approaching signif- 
icance was on the^ either-or questions, 
where, contrary to expectation, the 
moderately retarded were somewhat more 
responsive than the mildly^ retarded 
(t_ (28) = 1.83, p = .078). All dif- 
ferences between the severely retarded 
and the other two Qroups were signif- 
icant beyond the .01 level. 



Thus, the IQ group differences in 
total responsiveness for this sample, 
which pointed to deficiencies among 
the severely retarded, do not appear 
to be a function of the fact that 
most of the questions asked in the 
study were in yes-n~q form. The 
severely retarded appeared to be less 
capable of giving appropriate 
responses to all types of questions 
asked. Of course, their low respon- 
siveness . rates on verbal multiple 
choice and open-ejided questions, and 
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IN^ OTHER SAMPLES 

Table 5.8 presents the mean respon- 
siveness percentages for the same 
categories of questions (although not 
exactly the same questions) in the 
children's institution, adult insti- 
tution, and child community Scimples. 
One can readily see that the ordering 
of the question formats is much the 
same in all three Scimples. Among 
institutionalized adults, factual 
yes-no questions about activities 
were slightly easier than yes-no 
questions of a subjective nature, 
and both types of yes-no questions 
were significantly easier than verbal 
either-or questions, which in turn 
elicited higher responsiveness scores 
than either verbal multiple-choice 
or open-ended questions. Thus these 
findings paralleled almost exactly 
those for the institutionalized chil- 
dren, either-or questions being more 
difficult than yes-no questions but 
easier than the challenging open-ended 
and verbal multiple choice questions. 

The pattern of differences in the com- 
munity children's sample was somewhat 
different. Pictorial choice and fac- 
tual yes-no questions about activities 
were significantly easier than all-^ 
other types of questions. Subjective 
yes-no questions, while still falling 
next in order, were significantly 
more difficult than- these two formats 
but yielded higher responsiveness 
scores than did the remaining formats. 
Verbal either-or questions , rather 
^than being easier than verbal multiple- 
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Table 5.8: Mean Percentages of Appropriate Response to Various 
Types of Questions in Three Samples 





Sample 






Institution 
Children 
(N=50) 


Institution 
Adults 
(N=42) * 


Community 
Children 
(N-57) 




Question format 










Pictorial choice 


83.3 


(not used) 


96.1 




Verbal yes-no (factual) 


82.2 


84.8 


94.0 




Verbal yes-no 
(subjective) 


76 is 


81.7 


81.7 




Verbal either- or . 


68,7 


65.7 


72.5 




Verbal multiple-choice 


49.7 


51.9 


72.9 




Open-ended 


51.0 


50.6 


69.9 





*Excludes profoundly retarded groMO. 

choice and ooen-ended questions, 
proved to be as difficult. In this 
sample, responsiveness was generally 
high, and there was not the marked 
drop in responsiveness associated 
with verbal multiple-choice and 
open-ended questions bs there had 
been in the institutional samples. 

Still, the ordering of question for- 
mats must be considered highly 
~ stable from sample' to sample. One 
/ can generally expect to obtain 
answers from a large proportion pf 
a sample of retarded persons if one 
ask^ picture choice and verbal yes- 
no questions. Not coincidentally , 
both types of questions can be an- 
swered nonverbaliy (by pointing in 
the first case, by nodding or 
shaking the head in the second) . 



The use of verbal either-or questions 
involves some sacrifice of respon- 
siveness, but still can be expected 
to yield appropriate answers from 
roughly two-thirds of the sample. 
To our surprise, verbal multiple- . 
choice questions were consistently 
as difficult to answer as the more 
obviously difficult open-ended ques- 
tions. In our interviews, these 
questions typically required a choice 
among four, or sometimes three, al- 
'ternatives (e.g., "never," " not-^ 
much," "sometimes," or "a lot" in 
answer ,to a question about the ex- 
tent of involvement in some activity) . 
Apparently, either because a quanti- 
tative concept was involved or be-, 
cause subjects were unable to process 
so. many options, these questions 
Droved as difficult as questions that 
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required respondents to generate re- 
sponses - indepen ntly, with benefit 
of structure. 

It is difficult to fully convey the 
problems we encountered with open- 
en'ded questions without providing 
concrete examples. Samples 5.9 and 
5.10 provide verbat:.m answers given 
by virtually all members of the 
adult sample who cou3.4 respond ver- 
xl5a]ly in some manner. (We excluded 
"don't know" and "hull?" responses as 
well as a few repetitive answers.) 
The tables illustrate how the 
ability to answer such questions 
improves cis IQ increases / but they 
also illustrate how few responses 
to open-ended questions were inter- 
pretable. Many adults misunderstood 
the questions. For example, several, 
instead of indicating what people 
did that bothered- them, named people 
who bother' them, and some seemed to 
think that th)^ should discuss 
things they did that are nice or 
bothersome, or simply things tha - 
they like or dislike. Where re- 
spondents understood the questions, 
their answers did provide some clues 
to how they view the social world 
around them (e.g., note the comments 
about noise as a source of irrita- 
tion) , but the insights to be gained 
are limited. Particularly for ques- 
tions for which categories were con- 
structed after client and signifi- 
cant other responses had been 
reviewed, it proved extremely dif- 
ficult "to sort answers into meaning- 
ful categories. Moreover, as the 
tables suggest, once one omits the 
irrelevant or inadequate (vague) 
responses, ^v^ry few codable answers ^ 
are left to categorize in the first 
place. Again, while insightful 
answers sometimes femerge from more . 
verbal respondents, the return on 
investment, if we may borrow a 
phrase from business, is generally 
prohibitively low for open-ended 
question^. ^ 



What of the differences among IQ 
groups in their ability to manage 
different question formats? In the 
adult institution samples, differ- 
ences between IQ groups were not 
significant for subjective yes-no 
questions, factual yes-nq questions 
about activities, and multiple- 
choice questions, although trends 
pointed toward higher responsiveness 
as IQ increased. On both either-or 
questions and verbal multiple-choice 
questions severely retarded persons 
were significantly less responsive 
than moderately retarded persons. 
(These analyses included the mildly 
retarded subject who was generally 
unresponsive and who lowered the 
mean for mildly retarded subjects.) 
On open-ended questions, the severe- 
ly retarded group was even more 
seriously deficient, averaging 30.3% 
responsivenesS/Oompared to 62.7% for 
moderately retarded persons and 63-9% 
for mildly retarded persons- Thus, 
in this sample, while the severely 
retarded group invariably had the 
lowest mean responsiveness, their 
most notable deficiency wa^ on open- 
ended questions. 

Among the community children, a 
somewhat similar pattern emerged- 
While there were nonsignificant 
differences on yes-no questions 
about act^'vities, picture choice 
questions, and subjective yes-no 
^ questions, severely retarded subjects 
were significantly less responsive 
than both highelr IQ groups to verbal 
either-or questions. On verbal 
multiple- choice questions, they 
were significantly less responsive 
than were mildly retarded subject^- 
And finally, once again, the most 
striking ^IQ^group differences emerv7e'j 
for open-ended questions* Here, 
sevarely retarded respondents , with 
a mean responsiv?:\<*ss score of 51-1%, 
were significantly less responsive 
than the moderately retarded (68.9%), 
who in turn were less responsive 
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Sample 5,9 

"What Nice Things Do people Here Do for You? What Else?" 
Selected Responses of Institutionalized Adults* 

Profound 

It's from playing games on the record hop 
Yeah 
Severe ^ 
Nice 

Like to Work 
No way 
Work 

Mrs, . , she's my houseparent, 14,3. , etc, (list names) 

Three 
Mary 

I don't do that. /Sometimes you talk tes> people/ and if they go out, 
that all. And sometimes we go skating/Sometimes we dancer 
together. 

We talk, that's all. 

; i„used to play around and ride my bicycle. 

Nice 

^ Moderate \ ^ ' _ 

^. Play with me/play games/play hide and/ seek/farmer in the dell and 
hopscotch and play jacks/answar telephone 
They give me^ books and they give me necklace/give me home on weekends/ 
well, buy me things/I don' t think of no more. 
- I. do mysel^'help me out. 



Get me up in the morning and feed me my breakfast/They, help me get 
on the pot and off the bathroom/They feed me sometimes/They help 
me go to school/They're so nice to me/They treat me nice here. 
They let me go to the canteen when I want/Let me buy something when 

I want to/Come to cottage and drink a coke and stuff, take a bath 
and get ready for supper/Just go swimming whenever we want/Sweep 
during P.M. 
Let us go shopping/after movies/bowling 
Oh, not much 

Let me buff the floor/Mop 

They help you out/Show you how to do things/ (unintelligible) and make 
the bed 

They work here and clean up, mop and clean and, kiss. 
Give us treats . 

I heip in the kitchen/I take my plate up 
Nice peoples. They been good, quiet. 

Usually they're so tied up. But when they have.-time, I usually go 
someplace with them/I made my bed this morning 

Mild . J - _ 

Work/Stay work/wash face, hair/shave/try hair spray 
Help me out/Help me out when I'm sick 

They do, too — buy me things, everything I need/They let me do things I 
want to/They^let me go places 
/ Try to help me get a job/Help every people 

They help me walk/They go to the canteen for me 
V- They tell me when it's time to eat and go to class and sometimes we 
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just sit down and tell each other jokes 
Wash your clothes/Talk to you. Ask you questions, talk about things 
you want to talk about. -Look at things you want 'em to/Sew 
your clothes, talk to you, put things up. for you, pick things 
up for you/Remind ycv to do stuff 

Well, I help maKe beds and srrip beds and dress the girls. And if 

c — I 

I'm nice, I stay in the back and help ladies strip beds. If I 
stay back there, they don't have to yell at me and they let me 
have a cup of coffee for breakfast and they let me wash dishes/ 
When they're up in front, I'm usually washin' .and dryin' clothes 

when I lived in . .. (goes on to talk about punishments). 

Slashes mark points at which the probe, "What else?" was provided. 
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Sample 5.10 
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'What Do People Do Here That Bothers You? What Else? 



Selected Responses of Institutionalized Adults* 



Profound 



Yeah 



/ 

It's from bothers you, bother, bother 



Severe 




'Fight 

' i 

Did you find liy screw? (for his glasses) 
Nobody bothers me 

Roy bothers me. I buy him a coke, he's good (rambling on) 
^ Kevin , George , Jame s 
Nobody 

Alice does sometimes 

They do. I slap people/There's somethingV they do and I hit *em. 




Work 



Calls me 



and I don't like that namfe. 



Moderate 



Nothin 



Oh, some bother me. I get along with some, not all# 



Make noise/They yell/They keep it 



I hate to be picked on/noise/that's it 




They pass me around 



Fuss 



I don't like going ^around doing stuff /Everything bothers me/Pickin up 



trash. 
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Knock in the head/Knock down; soup when We have soup 

Gets you in trouble/and play 

Nothing, noise don't hurt me. . 

Chews tobacco and stuff /smoking/pestering when I. get off from laundry/ 
I can rest.' I'm tired when I get home from work but they disrupt 
me/stealing, I can't stand that/ that's it 

Mild 

Saturday. 

Nothing - 
Bad man 

Call names, cuss; they cuss and talk mean/ (unintelligible) /stare at 
you/they kick you every day 

Talk a lot/when I try to sleep, they wake me up. That bothers me more 
than anything in the world. ^ 

I don't like real loud talk. I don't like fussin' or fightinV 

hittin' people/one thing they have on my medical folder, that 
' ' whenever I was born something happened to my mind. They say 

they can't get my attention as fast as they want to. I really 
want to go home for good. I believe I'm old enough. 

They hit on me and they call me names and they bite me and they get 
me in trouble/nearly every morning, every afternoon and every 
night, I have to stand this screamin' and yellin' that goes on 
every afternoon, morning, and night, and i get this headache 
that won't go away. I called the housemother and asked her to 
ask the nurse to give me a couple of aspirins. But the only 
person in this cottage who gets aspirin is . 
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Sometimes they might ask questions I can't answer, or a few other 
things. I don't get bothered real easy-nothing really. 
♦Slashes mark points at which the probe, "What Else"," was provided. 



\ 



5.22 



9[) 



4J 

O 

U 

C 

! 

in 

4J 
C 
0) 
U 
U 
0) 



100^ 



90- 



80 - 



70- 



60- 



50- 



40 - 



30- 



20- 



10 - 




Yes-No 

Either-Or (Pictures) 



Open-ended 
Either-Or (Verbal) 
Multiple Choice 



Community 
Children (N 



= 57) 



X 



Severe Moderate Mi Id 

Lievel of Retardation 



Figure 5.2 Responsiveness to Various Question Types as a Functionof Level of Retardation 



than the mildly retarded (88.5%). 
These findings are strikingly por- 
trayed in Figure 5.2. 

The pattern of results, then, sug- 
gests that while severely retarded 
persons can generally be expected to 
have the most difficulty responding 



to questions, their handicap is most 
evident on the questions which can 
be considered the most cognitively 
demanding, particularly open-ended ' 
questions which put the burden for 
generating an answer on the inter- 
viewee. These findings, collectively, 
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have clear implications for survey 
design where the goal is to optimize 
responsiveness. We will consider 
another means of optimizing respon- 
siveness next. 

Effects on Responsiveness 
of Asking Questions Twice 

Certainly one possible reason for 
low responsiveness is the inter- 
viewee's failure to process the ques- 
tion. Throughout our research, we 
used responsiveness to the first 
asking of the question as the major 
measure of responsiveness. However, 
we also used the strategy of asking 
the question twice if the inter- 
viewee had not been able to answer it 



on first asking. Moreover, in the 
adult institutional sample, we ex- 
plicitly tested the value of repeat- 
ing the question as a way of increas- 
ing responsiveness. 

The analysis focused on a total of 131 
different questions from both forms 
which were asked of all interviewees. 
The measures compared were the per- 
centage of the sample responding with 
at -least, a minimally appropriate . 
response after both first and second 
asking. These two figures were com- 
pared through t tests for correlated 
means for each of five sets of ques- 
tions: 82 yes-no questions, 20 open- 
ended questions, 11 multiple^choice 
questions, 13 picture choice ques- 
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tions, and 5 either-or questions. 
Figure 5,3 shows the mean respon- 
siveness figures for each of these 
five sets at first asking and for 
first and second asking combined. 

The figure clearly indicati^.s that 
for each category of question there 
was at least some gain in sample 
responsiveness figures associated 
with giving interviewees a second 
chance to answer the question. In 
fact^ despite low numbers of ques- 
tions in some question sets ^ all of 
the t tv-sts were significant at the 
.01 level at least. In terms of 
sheer increase in responsiveness, 
the gains were most striking on 
either-or and multiple choice ques- 
tions, where the percentage of the 
sample ultimately responsive was 
approximately 11 percentage points 
hj.gher than the percentage respon- 
sive at first asking. The gains 
for picture choice and yes-no ques- 
tions were moderate by comparison 
(3.8% and 5.4%, respectively), but 
this may^'-very well have been due to 

^ fehe^fact that responsivene;^ s to the 

first askings of these questions 
was already almost at its ceiling. 
(Although roughly two-thirds of the 
sample appears responsive in 
the figure, it must be noted that 
these figures are for all 58 sub- v 
jects, 14 of whcm had failed the 
screening, interview and, thus were 
defined as totally unresponsive. 
Thus, if all interviewees who actu- 
ally received the entire interview 
had answered a question, the respon- 
siveness figure would have been 
approximately 76%.) Open-ended 
questions were difficult on first 
asking, and the gain attributable 
to asking the question twice (about 
eight percentage points) was not as 
large as that for the almost equally 
difficult verbal multiple-choice 
questions. By analyzing specific 
ciianges, we found that persons who 
had given an inadequate (vague) re- 
sponse the first time were most 



likely to give an appropriate re- 
sponse the second time, but that a 
variety of inappropriate respgnses 
could be converted to appropriate ones 
if the question was repeated. 

Summary and implications 

As the first challenge in inter- 
viewing mentally retarded persons is 
to obtain a uscible answer of some 
kind, we devoted considerable energy 
to measuring and analyzing respon- 
siveness to the communication demands 
of questions. The nine-category 
coding system we developed proved to 
have higher interrater reliability 
and to adequately classify the range 
of possible behaviors in response to 
questions. There are, to be sure, 
problems in coding responsiveness. 
For example, -the judgment that a. 
response is unintelligible is clearly 
a function of interviewer sensitivity; 
"don't know" responses under some 
conditions should probably be labeled 
appropriate rather than inappropriate 
responses; and there are judgments 
to be made in determining that an 
answer is too vague to qualify as an 
appropriate response, especially 
when, for example, subjects give 
relatively sophisticated but vacil- 
lating answers to yes-no. or either-or 
questions which ultimately cannot be 
construed as taking one side or the 
other. Still, the responsiveness 
coding system generally served its 
purpose of providing a means of : 
examining answering behavior independ- 
ent of the reliability or validity 
of answers. 

As indicated in this chapter, we 
discovered several important things 
about responsiveness. First, the 
reasons for failure to respond ap- 
propriately are diverse. In inter- 
viewing retarded persons, one can 
expect silence in response to ques- 
tions' as well as various kinds of 
unintelligible,^ vague, and totally 
irrelevant responses. As indicated 
by the predominance of minimally 
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appropriate responses as compared 
to expanded appropriate responses, 
one can also expect codable answers 
to generally be brief, in many cases 
monosyllabic. 

As our analyses clearly demonstrate, 
responsiveness is a stable individual 
characteristic. An individual's 
ranking in the group based on the 
percentage of questions answered 
appropriately is remarkably stable 
from interview to interview. While 
some subjects appear to become more 
responsive to a second interview as 
a function of having participated 
previously in an interview, changes 
in responsiveness from administration 
to administration were slight. More- 
over, responsiveness as a behavioral 
characteristic appears to be related 
to intelligence. Certainly we found 
that verbal interviewing techniques 
are infeasible with the vast majority 
of profoundly retarded persons. In 
the severe range of retardation, 
individual differences were very 
large, and it was difficult to pre- 
dict which individuals would^ be able 
to respond. This suggests that 
screening potential subjects for 
their ability to participate in an 
interview, possibly with more strin- 
gent criteria than were used in our 
studies, is especially useful in the 
severe range of retardation. Other- 
wise, one can generally expect most 
moderately and mildly retarded per- 
sons to be able to answer most ques- 
tions asked in a simple survey. Our 
data also indicated that while 
responsiveness was a function of 
IQ,- it w^s unrelated to the sex,, 
age, or place of residence (insti- 
tution versus community) of the 
retarded person. 

Perhaps the most practically impor- 
tant finding about the nature of 
responsiveness is that it depends 
not only on the subject' s cognitive 
abilities but on the form in which 
questions are asked. Judging from 
the consistent findings across three 



samples, we oan confidently conclude 
that yes-no questions and questicgis 
calling for a choice between pic- 
tures are the question formats likely 
to optimize responsiveness. Verbal 
either-or questions also appear to 
qualify as an attractive option if ^ 
obtaining codable answers from a major 
ity of a sample is the goal. However, 
verbal questions requiring choice 
among three or four alternatives and 
open-ended questions appeared to be 
difficult to answer, particularly 
for the severely retarded. As we 
will point out in later chapters, 
there were other problems associated 
with these formats besides low 
responsiveness, and altogether they 
prove to be relatively poor sources 
of information in most cases. *At 
the very least, it must be recognized 
that many persons in a sample will 
simply not be heard from when these 
formats are used and that their 
utility is greatest when interviewees 
have the high verbal ability required 
to comprehend them and generate 
answers.. In keeping with these 
findings, one possible approach to 
survey design would be to construct 
layered interview schedules so that 
each area .of questioning begins with 
the relatively easy-to-answer struc- 
tured, questions, especially those 
which can be answered nonverbally, 
and then progresses, where the inter- 
viewee's communication skills permit, 
to less structured questions which 
can then provide more specific 
information to clarify closed-ended 
responses . 

Finally, our data suggest another 
relatively simple way of increasing 
responsiveness; that is, repeating 
questions, possibly even more than 
one'ti'.me if necessary, in. order to 
give retarded persons additional 
opportunities to understand them. 
The simple technique of repeating 
a question if the first response was 
less than appropriate did increase 
responsiveness for all types of 
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questions examined. If simple repe- 
tition works, what about the common 
practice of rephrasing questions to 
make them more understandable? We 
consciously avoided this practice 
in our research because we did not 
want to run -he risk of altering 
responses by altering questions. 
As our later findings will illus- 
trate, the way in which a question, 
is worded or formatted can make a 
substantial difference in the 
answers that are given. Unpremedi- 
tated rephrasing of questions is 
particularly dangerous, for people 
have a natural tendency to simplify 
the form of the question; for 
example, dropping back to what 
appears to be a simpler and more 
structured format if an open-ended 
format does not work. Thus, when 
an interviewee fails to answer 
appropriately the first time, we 
would recommend starting with a 
verbatim repetition of the question. 
If that does not work, responsive- 
ness might still be increased 
through the use of preplanned 
rephrasing of questions c However, 
we would recommend that these re- 
phrasings involve changes in wording 
rather than changes in format simply 
because changes in format, while 
they may be more likely to increase 
responsiveness, are also more likely 
to alter the content of responses, 
judging from evidence to be pre- 
sented later. 

Finally, we might add a word about 
the implications of our responsive- 
ness data for the communication 
development ^f^ mentally retarded 
persons. The predominant trend in 
the previous literature on the com- 
munication skills of the retarded 
has been to study the . forms of lan- 
guage and the development of the 
ability to comprehend and produce 
those forms. Recently the emphasis 
in the study of normal language 
development has shifted so that 
researchers are more concerned with 



the use of linguistic forms in, var- 
ious situations, with the functions 
that language serves in communication 
settings. The measure of responsive- 
ness used here is a measure of behav- 
ior in a real communication setting? 
more precisely, a measure of an in- 
terviewee's tendency to meet the 
communication demands of interview 
questions. Although ours is not a 
developmental study, it suggests that 
there is a developmental sequence 
that children must progress through 
in understanding and answering ques- 
tions- Like some previous research 
with normal children (e.g. , Ervin- 
Tripp, 1970) it suggests that struc- 
tured questions may be mastered 
earlier than unstructured questions 
and 'suggests more specifically which 
kinds of questions can be answered 
most easily by persons with low 
mental ages. We do not yet know the 
extent to which responsiveness to 
questions of increasing difficulty 
can be. trained. However, we are, in 
the context of a related study, 
testing the effectiveness of such 
training. It would appear to us 
that since answering questions is a 
common form of verbal behavior, 
explicit training in responsiveness 
should have an important place in 
language arts curricula for the re- 
tarded. Such training might first 
aim to establish minimally appro- 
priate responses and then encourage 
-Students to go beyond the minimal 
demands of the question to qualify 
and elaborate answers and to antic- 
ipate upcoming questions. Other 
research we have conducted (e,g., 
Elias, Sigelman, & Danker-Brown, 
(1980) suggests that the ability of 
retarded adults to answer questions 
appropriately is associated with 
making positive impressions on 
raters of videotaped interviews. 
Thus, there is some evidence that 
training to increase responsiveness 
to questions might not only con- 
tribute to communication development 
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but facilitate -social and vocational 
acceptance of retarded persons. 

To this point in the narrative, we 
have considered only the ability tc 
respond to questions. However, ob- 
taining an answer appropriate to the 
ques'tions is only part of the battle. 
One might ask whether the answers 



obtained are reliable and valid if 
one is to have confidence using 
interviewing techniques as a source 
of information about the circum- 
stances, needs, and attitudes of 
retarded persons. In the next 
chapter, we confront the reliability 
issue. 
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Chapter 6 



TO WHAT EXTENT ARE ANSWERS RELIABLE 

OVER TIME? 



/ 



Assuming that an answer is 
obtaine ' from a mentally retarded 
interviewee, one must immediately 
be concerned with the reliability 
amd validity of response. This 
chapter concerns reliability of 
response or the extent to which 
answers can be considered stable 
indications of interviewees' needs, - 
circvun'stances, and attitudes. There 
are at least two ways in which the 
reliability issue can be approached 
in survey research. One approach, 
a variation on test-retest relibil- 
ity, involves readministering the 
same questions after a brief interval 
of time and determining the extent 
to which answers given on the two 
occasions are consistent. The second 
approach is akin to measuring test 
reliability by determining the 
consistency of performance on alter- 
nate forms of a test. It would 
involve asking questions in alter- 
native ways cmd determining the 
extent of agreement between responses 
to the alternative questions. In 
the present study, we did indeed 
ask a number of questions in alter- 



nate forms, but we have chosen to 
discuss the results in the context 
of validity rather than reliability. 
Perhaps this decision could be 
debated, but in any event, the present 
chapter's discussion of reliability 
of response is concerned only with 
the extent to which answers given 
one week agree with answers given by 
the same respondents the next week. 

Two interviews were administered 
approximately a week apart to both 
institutionalized children and insti- 
tutionalized adults. In each sample, 
half of the siibjects had Interviewer 

1 first and Interviewer 2 second, 
while the other half had Interviewer 

2 first and Interviewer 1 second. 
Since two forms of the interview 
schedules were used (and were also 
counterbalanced for order of presen- 
tation) interviewees were, exposed 

to somewhat different interview sched- 
ules on the two occasions. Within 
those schedules, however, some ques- 
tions remained in identical form on 
both occasions, and the only differ- 
ence was the .context of other 
questions. 
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There is no absolute criterion which 
can be used to make the judgment 
that responses are reliable or unre- 
liable. As we shall point out, the 
analysis of such information is very 
complex, and multiple guidelines for 
interpreting levels^of consistency 
may be called for. One can, in 
looking at the literature on test 
reliability, generally point to a 
criterion of reliability of .80 to 
.85 or higher, but this criterion is 
not typically applied to the relia- 
bility pf individual test items so 
it serves only as a rough standard. 
We will indeed examine the percentage 
of paired rcisponses that are consis- 
ent rather than discrepant. As we 
will indicate, afiother perspective 
can be obtained by examining ^-'-^e 
extent to which the degree o-' consist- 
ency obtained deviates fxom the 
degree of consistency which could be 
expected on the basis of chance, but 
this criterion alone is not entirely 
suitable either. To complicate ' 
matters further, one must consider 
whether the data obtained through, 
a survey are to be analyzed at the 
aggregate or the individual level. 
It is entirely possible that many 
individuals might change their 
responses from one. occasion to t^e 
next but that the overall picture of 
the group obtained will not change 
a great deal, assiaming that changes 
by individuals essentially cancel 
each other out. In this same situa- 
tion, however, it would be unwise to 
use the ^ata to identify individual 
correlates of response or to make 
program decisions affecting individ- 
ual clients. 

With that in mind, let us look in 
some/detail at the extent to which 
answers given by institutionalized 
chi/ldren were consistent from one 
we4k to the next. 



Reliability Over Time- 
Institutionalized Children . 

In the children's institution 
sample, 31 questions repeated on both 
forms were examined for reliability. 
^Thirteen of these were verbal yes-no 
questions about activities, quality 
of life, and rules and decision- 
making. An additional four questions 
of the yes-no variety were accompa- 
nied by pictures illustrating the 
chores asked about. fable 6.1 pre- 
sents a variety of information about 
these 17 questions and can serve to 
illustrate the issues in analysis of 
reliabilitsy . 

Generally a relatively high propor- 
tion of the sample was able to 
respond to both askings of each of 
these yes-no questions, although, 
responsiveness fell off somewhat on 
the rules and decision-making ques- 
tions , presumably due to the diffi- 
culty of such concepts as "decide" 
and "allowed." Thus, the figures 
are based on a fairly wide range of 
subjects (more so jthan is the case 
fot open-ended questions and other 
more difficult foijmats) , For the 
entire^ set of 17 questions , an aver- 
age of 86.9% of the respondents were, 
able to respond consistently "yes" or 
"no" to both askings. The percentages 
of consistent response, as shown in 
the fourth columni of the table, 
ranged from a low, of 56 . 1% on the 
question about wh.'ether residents are 
allowed to hit people to a high of 
100,00% on the qijestion about cleaning 
•the floor. Generally, the percentage 
of subject?; ^espi^nding to both ques- 
tions who were consistent in their 
responses were lowestHJI^questions to 
which a "yes" answer might be con- 
strued as a socially undesirable 
response. The questions in which a 
"yes," response would indicate that 
residents are allowed to hit people, 
that people yell or say mean things, 
and Wat people .take things away all 
had i^ow percentages of consistency. 
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Table 6.1: Reliability of Response of Yes-No Questions: Institution Children 
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Question 


Responsive 


(Form A] 


(Form Rl 


% CnrmicjtGnf" 




Do you watch TV? 


42 


97.6 


97.6 


95.2 


-.02 


Do you listen to the radio or record player? 


41 • 


92.7 


90.2 


92.7 


.53 


0,0 you go to school or take classes? 
Do you nave a family? 


41 


95.1 


92.7 


92.6 


.36 


44 


95.5 


100.0 


95.5 , 


.00 


Is this place your- home? 


40 


55.0 


67.5 


92.5 


.83** 


Do Deonle here veil at vou or sav mean thinas? 


38 


55.3 


44.7 


73.6 


.48** 


Do DGonle here take thinas awav from vou*^ 


43 


44.2 


48.8 


76.8 


.54** 


Do people here help you when you want help? 


. 39 


94.9 


92.3 


97.4, 


.78** 


Do people here teach you things you want to learn? 


40 


92.5 


. 95.0 


92.5 


.35,. 


Are you allowed to stayHp late at night? 


,33 


54.5 


54.5 


87.9"V 


;25** 


Are you allowed to hit people? 


41,, 


12.2 


56.1 


56.1 


.19 •■ 


Do you decide what chores you do? 


24 


58.3 


62.5 


70.8 


.39* 


Do you decide how to spend your money? 


36 


88.9 


88.9 


94.4 


.72* 


(With Pictures) 












Do you set the table? 


43 


97.7 


93.0 


90.7 


-.04 


Do you do dishes? 


44 


90.9 


.79.5 


84.1 


.38 


Do you do laundry? 


45 


73.3 


66.7 


'84.4 


.63** 


Do you clean the floor? 

\ 

'. \ 


43 

\ 

1 


95.3 


95.3 


.100.0 

- > 


1.00* 



* significant at the .05 lever (one tailed) ** significant at the .01 level. 



Consistency for the question about 
decision making regarding chores was 
also low, possibly in this case 
because two difficult concepts, 
"decide" and "chore," may have cre- 
ated confusion. Generally, consist- 
ency figures for the 13 verbal .ques- 
tions (average consistency 86.0%) 
were similar to those for tne ques- 
tions accompanied by pictures aver- 
age consistency, 89.8%). Generally 
the predominant form of consistency 
on most of these questions (with 
the exception of the negative itemr 
discusse'd above) was saying "yes""^^ 
both. times rather than saying/ "no" 
both times- "Yes" answers predomi- 
nated on most of these questions. 
Moreover, the percentage of subjects 
saying "yes" on Form A (£ = .88) 
as well as on Form B (i_ = .77) . 
This suggests that reliability is 
artificially inflated when marginal 
distribution is lopsided. As will 
be noted later, acquiescence was 
common in our samples; thus we sus- 
pect that the reliability figures for 
yes-no questions are generally 
inflated. We would at least warn 
that the relatively high consistency 
figures in Table 6-1 are not neces- 
sarily evidence of validity. However, 
we cannot estimate precisely the 
■extent to which acquiescence might 
have contributed to consistency. 

In any case, the figures are encour- 
aging in the sense that we can expect 
much the i same response both times 
from most subjects most of the time. 
As the second and third columns of 
the table indicate, this means that 
the pictures of the group obtained 
from two, askings of the same ques- 
tions are highly similar, the biggest 
discrepancies appearing on the ques- 
tion about being allowed to hit peo- 
ple and on the picture questions 
about dishes and laxindry, both of 
which were associated with only mod- 
erately high consistency figures.' 

A different perspective on relia- 
bility emerges from the last column 



of the table, which reports the 
kappa statistics calculated from 
each 2x2 contingency table analyzed. 
Kappa is a straightforward estimation 
of the extent to which the obtained 
per<?entage of consistency between 
paired responses exceeds the percent- 
age which might be expected based 
on chance, chance probability being 
calculated from the marginals of the 
contingency table. Assume, for 
example, that in the sample 90% actu- 
ally watch TV and 10% do not, and 
that 90 and lO are the marginal per- 
centages both times we ask the ques- 
tion. Now assume that we randomly 
pair first responses and second 
responses rather than pairing two 
answers given by the same respondent. 
As discussed in Chapter 3, we can 
calculate the probability of agree- 
ment between pairs of response by 
chance from the marginals, multi- 
.'plying .9 x .9 and .1 x .1 and adding 
the two to yield a probability of .82. 
This means that 82% of the pairs of 
response would agree simply by chance. 
When response distributions are lop- 
sided like this (e.g., when there 
is not approximately a 50-50 split 
between "yes" and "no" responses at 
each asking) the cliances of agree- 
ment become relatively high as illus- 
trated in this example. Thus the 
obtained percentage of agreement, 
even if it is relatively high, may 
be no higher than what we would 
expect if we simply paired by random 
assignment one subject's response 
with another subject's response 
instead of pairing two responses 
given by the same subject. (For 
those whose memories of statistics 
are jogged by the image of red and 
green balls in a jar, the probability 
of drawing two red balls becomes 
progressively higher as the' number 
of red balls as compared to green 
balls in the jar increases.) In 
interpreting kappa statistics then, 
we must bear in mind this question: 
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To what extent are people responding 
so consistently to the question both 
times that their consistency is 
greater than we would expect by 
chance? As can be noted in the 
table, several questions which have 
very high consistency figures do not 
have large or statistically signif- 
icant kappas, while several ques- 
tions with much more modest cor>sis- 
ency figures do have significant 
kappas. When a question yields an 
extremely high proportion of "yes" 
responses on both askings, it becomes 
very difficult to obtain a large 
kappa simply because the odds of 
agreement by chance alone are also 
high- - 

We are not willing to say that kappas 
should be used as the ultimate cri- 
terion of reliability. We consider 
it impressive to obtain 90% or 
greater consistency figures even if 
those figures are not a significant 
departure from chance expectations. 
Such high figures indicate that the 
picture of the group is reliable and 
that individuals within the group 
are saying essentially the same 
things on two occasions a week apart. 
On the other hand, we can be even 
more satisfied when the consistency 
figure and the kappa associated with 
it are high, as in the l^st question 
' in the table about cleaning the floor. 
Similarly, we can be particularly 
suspicious of the utility of a ques- 
tion if the consistency figure is low 
and the kappa is low as well, as is 
the case for the notably ineffective 
question about whether residents ar^e 
allowed to hit people. In between,' 
we have softer ground to stand on 
and must take into account all the 
information obtained, the percentage 
saying "yes" on each occasion, the 
percentage of responses that are 
consistent, and the extent to which 
this percentage deviates from chance 
expectation. . 

Overall, we would conclude that. yes- 
no questions appear to yield fairly 



reliable data.' JHowever, we would 
reemphasize the warning about acquies- 
cence, because saying "yes" on two 
occasions does not necessarily mean 
that "yes" is the valid response. 

Six multiple choice questions 
repeated across the forms also posed 
interpretive problems. Three of 
these questions which asked subjects 
to identify the extent of their activ- 
ity as "a lot," "sometimes," "not 
much," or "never" (or in one case 
"a lot," "some," "not many, " and 
"none" in reference to number of 
friends) yielded 71.9% consistency 
on the average. This figure is rela- 
tively high considering the fact that 
the probability of matching responses 
by chance is lower for a four-choice 
question than for a two-choice ques- 
tion. Indeed the kappas for these 
items ranged from ,51 to ,64 and 
were all significant at the .01 level. 
However, three ether questions, 
asking respondents how they felt 
about aspects of institutional life, 
yielded extremely low consistency 
(46.2 on the average), even though 
the kappas for two of the questions 
(.27 and .20) were statistically 
significant. It is impossible to 
determine whether this was the case 
because of the subjective and mood- 
dependent nature of the questions or 
because a greater proportion of the 
sample was able to respond to these 
pictorial questions than to the -ver- 
bal multiple choice questions^. In 
any case, this format cannot be con- 
sidered successful, / 

The remaining eight questioiYS 
repeated on both forms were /open- 
ended. Four were factual >^ nature, 
calling for: name, spelling ^f name, 
birthday, and characterization of the 
state school, . Roughly two- thirds of 
those asked these questions were 
responsive to the first ahd second 
administrations of each; of these 
63.3% were consistently able to give 
their first and last names correctly? 
42,9% were consistently able to spell 
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their names; 63 ,6% consistently iden- 
tified their birthdays; and 61.9% 
consistently described the place in 
which they lived. This finding is 
rather discouraging, not only because 
many persons could not answer these 
basic questions, but also because 
those who did often gave incorrect 
information both times- On the other 
hand, the ari^Eeria for correctness 
were rather strict. 

The remaining four open-ended ques- 
tions were each analyzed by response 
categor^ies, which were based on 
responses given by both/ residents and 
attendan-bs. in these cases, both the 
total percentage cQiisistent (i.e., 
either mentioning an item in a cate- 
gory on both askings or failing to 
mention it both times) and the per- 
centage mentioning something both 
times were examined. There were 
marked differences between the two 
measures. On a question asking for 
an enumeration of activities with 
friends, for example, the average 
total consistency figure across all 
eight categories was 70.7%, but the 
average percentage of respondents 
consistently mentioning something 
both times was only 18.1%. Virtually 
the same thing occurred for the other 
questions asked. For example, asked 
about what they were being taught in 
school, 69.6% of those who could 
answer both times referred to academ- 
ics vo^ "some kind, but for each other- 
category of response^ high total con- 
sistency was achieved by failing to 
refer to a category both times. 
There seems to be no satisfactory 
way of analyzing such questions, for 
consistency depends on both the 
verbosity of the respondents and the 
niomber of categories developed by 
the coders. Perhaps the major con- 
clusion to draw is thai: open-ended 
questions calling for enumerations 
yield very little information, par- 
tially because many mentally retarded 
persons cannot respond appropriately 
to them in the first place, and par- 
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tially because when they do respond, 
they say very little ♦ Due to the 
predominance of "no mentions," only 
9 of 35 kappas were significant. 
However, since the total consistency 
figures for all 35 response- cate-^ 
gories of four questions averaged 
83.9%, one can at least conclude 
that it was relatively rare that a 
respondent mentioned something one 
time but failed to mention it \the 
other time. 

Judging from this sample, reliability 
of response from one week to the next 
can generally be considered margin- 
ally adequate, except for the subjec- 
tive yes"-no questions mentioned pre- 
viously about rules and decision 
making and the picture-choice ques- 
tions about the quality of institu- 
tional life. Reliability did vary 
as a function of both question format 
'and question content, but for the 
most part answers to questions 
repeated a week apart were largely 
similar. 

Reliability Over Time — 
Institutionalized Adults 

In the adult institution sample, 
48 questions were repeated on both 
forms and could be examined for 
reliability. The questions were 
analyzed in three separate groupings 
according to question format: yes-no, 
either-or, and open-ended. 

The 29 yes-no questions were divided 
into those which elicited factual 
information and those which required 
a subjective response from the inter- 
viewee, with the expectation that 
reliability might be higher for 
factual questions. Analysis of re- 
sponse consistency across both sur- 
vey forms revealed that the average 
consister^ for yes-no questions was 
81.2%. /Contrary to expectation, 
there was ^almost no difference in 
consistency between the factual and 
the subjective questions (80.6% and 
81.7%, respectively). Response con- 
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sistency across this set of questions 
ranged from 97,3% to 60.5%. The 
highest consistency figures were 
found for questions like "Are you 
usually happy?" Do you have a family?" 
and "Do YOM know how to cook?" 
while the lowest consistency figure 
(60.5%) was obtained on the question 
"Are you usually sad?" another question 
for which "yes" can be considered a 
socially undesirable response. 

There was a definite tendency for 
the majority of interviewees to con- 
sistently give "yes" responses to 
many questions (21 of 29) . It ap- 
peared that the reliability figures 
for this sample, as was the case for 
institutionalized children, were 
inflated by a tendency to acquiesce 
on yes-ndx questions. Reliability 
was higheXt £or those items for 
which "yes"^Was a socially desirable 
choice. However, even if such a 
response set were not a significant 
factor in the high reliability fig- 
ures obtained, it cannot be assumed 
that consistent responses to these 
yes-no questions necessarily repre- 
• sent accurate or valid information. 

Thus the picture of this sample 
based on their responses to these 
questions appears to be primarily 
positive, with the majority of the 
interviewees seeing themselves as 
usually happy or possessing a niamber 
of basic adaptive skills such as 
reading, writing, keeping house, 
counting money, and cooking. This 
picture may have been influenced by 
a tendency to acquiesce or to give 
socially desirable responses as al- 
ready discussed, and indeed we will 
denionstrate later that attendants 
are not so ready to agree that 
clients have basic adaptive skills. 
However, the responses' to questions 
such as, "Are you usually sad?" and 
"Do you have any problems?" seem to 
suggest that many clients can say 
"no" to deny having difficulties. 



There were four verbal either-or 
questions repeated on tne two alter- 
nate surveys consisting of two pairs 
of questions Vi-th the order of op- 
tions varied: "Are you usually happy 
or sad?" vs. "Are you usually sad or 
happy?" and "Are you usually by 
yourself or with other people?" vs. 
"Are you usually with other people 
or by yourself?" Average consist- 
ency for these four questions was 
again relatively high (82.2%) and was 
comparable to the average consistency 
obtained for yes-no questions. A 
substantial majority, of respondents 
consistently indicated that they were 
usually happy ir;! response to both 
happiness questions. However, there 
was a tendency for more interviewees 
to consistently answer "sad" when this 
alternaitve was given second (12.0%), 
than when it was the first option (2.9% 
consistently reporting "sad"). 

/ 

The relationship between the two 
• questions of the second either-or 
pair was more complicated, although 
a similar pattern emerged. The 
majority _o£ respondents consistently 
replied to both questions that they 
were usually with other people as 
opposed to being by themselves. As, 
with the previous question pair, the 
percentage who consistently chose 
the alternative "with other people" 
was slightly higher when this choice 
was presented as the second option 
in the question than when it was 
first (50.0% vs. 46.7%). In 
addition, the percentage who consist- 
ently chose the "by yourself" alter- 
native doiabled when this option was 
presented last (33.3% vs. 16.7%). 
For this question, "Are you usually 
' with other people or by yourself?", 
the effect of consistently choosing 
the last alternative was to increase 
the overall percent of respondents 
who answered the question consist- 
ently both times to 80.0% for this 
question as compared to 66.7% for 
the alternative question. The obser- 
vations made from these two either- 



or question pairs seem to indicate 
that a small portion of interviewees 
are probably influenced more by the 
order of the either-or alternatives 
than by the actual conteijit of the 
response alternatives. 

The last 15 questions on both surveys/ 
were open-ended. The first seven / 
open-ended questions asked the resi-/ 
dents to give f actual \^informati on / 
concerning their names^^, birthdates,/ 
and current addresses. Responses to / 
these questions were c6c?ed either / 
correct or incorrect based on records 
kept on the subjects. ' On the average, 
51.9% of the residents who responded 
to be questions consistently gave 
correct answers^ Residents wer^ most 
likely to give correct information 
about their dates of birth: 77^4% 
correctly identified the month they 
were born in, 72.2% consisten/:ly gave 
the correct date, and 58,8% were able 
to give the right year both ^times. 
In comparison, only one- third (33.3%) 
,,of the residents could correctly 
' give, their full present address con- 
sistently, while slightly/ more than 
half (56.7%) were able to correctly 
give their first and last names at 
both interviews (only'^irst or last 
names were scored as' partially 
correct). Only 25.0% of the resi- 
dents could consistently spell their 
names (orally) when asked, making 
this the most difficult question of 
the seven. ^ A greater nximber of sub- 
jects (40.0%) were able to cor- 
rectly write their names both times.. 
Five of the 15 were coded with pre- 
sent response categories , and the 
average consistency for these ques- 
tions was 83.3%. This figure would 
have been higher except that response 
consistency for one question, "What 
. kind of place is this?", was excep- 
tionally low, with 55.0% giving con- 
sistent descriptions and only 30% 
consistently identifying the place 
where they lived as a state school. 
A clear majority of the respondents 
consistently indicated that someone 



other than themselves made 'the rules 
where they lived (92.0%) and decid- 
ed what chores they would do (75.0%). 
In contrast, the respondents were 
almost equally divided on the ques- 
tion "who decides how you spend 
your money?", with 50.0% consistently 
answering "someone else" and 43,8% 
consistently indicating that they 
made the decision themselves. Total 
reliability figures for these three 
questions were uniformly high: 96.0%, 
83.3%, and 9*3.8%, respectively. 
These findings indicate that most 
subjects who can answer giVe reliable 
answers to open-ended questions con- 
cerning who makes decisions about 
their everyday activities. Inxaddi- 
tion, 90.8% of the subjects con^st- 
ently answered the question, "Howv 
many people sleep, in your bedroom?\ 

The remaining three open-ended ques-- 
tions assigned each resident's an- 
swer to one of several response cate- 
gories made up on the basis of the 
responses given by both residents 
and attendants. The average consist- 
ency for this set of 26 open-ended 
response categories was 87,4%, the 
highest average consistency for any 
of the question groups. analyzed. - 
However, the consistency percentages 
for the response category questions 
were inflated by the large number of 
categories whose high consistency 
was achieved by clients' repeatedly 
failing to mention anything in the 
category. For example, on one ques- 
tion Which asked the resident to 
name the person who helps him/her 
the most, there was only one category 
(cottage staff) for which the most 
-common vresponse combination was men-^ 
tioning' that category both : times. 
For the other eight response cate- 
gories for this questic" the most 
common response combination was not 
mentioning the category either time. 
Therefore, while the average total 
consistency figu^ across all nine 
response categories was high (81.5%), 
the average percentage of . respondents 




consistently mentioning something 
both times was very low. 

This same pattern was found for the 
remaining two response category ques- 
tions. When residents were asked 
who they usually talked to about 
their problems, the most consistently 
mentioned response category was "cot- 
tage staff (36.4%). Four out of the 
remaining six categories were men- 
tioned by only a few residents, 
making the average percent of respond- 
ents mentioning a category both times 
again quite low (9.1%) . On the final 
question of this group, "If you had 
one wish, what would you wish for?" 
no response category was mentioned 
very frequently on both occasions. 
The most frequently and consistently 
mentioned wishes were to be with 
their families (27.3%) and to have 
material items (13.6%). Consistency 
in responding to this question was 
achieved primarily by failing to men- 
tion a response category both times. 
Thus open-ended questions , which 
could not be. answered by many sxibjects 
in the first place, were reliably 
answered, but rarely generated con- 
sistently mentioned information. 

overall, for this sample of insti- 
tutionalized adults, response con- 
sistency for questions repeated at 
a one-week interval averaged over 
■80%, providing interviewers with 
\answers that were reasonably reli- 
able. There was little variation in 
response consistency associated with 
4he three different question formats, 
H^pwever, there was some indication 
that high consistency on yes-no 
qiiestions may have been gained in 
pairt through acquiescence, that some 
sxibjects prefer the second alterna- 
tive on either-or questions, and 
that consistency on open-ended ques- 
tions was gained primarily by fail- 
ing to mention something both times 
rather than mentioning something 
twice . 



Individual Correlates of Response 
Reliability 

As was done in analyzing respon- 
siveness as an individual character- 
istic, an analysis was conducted to 
determine what accoxants for individ- 
ual differences in reliability of 
response from one week to the next. 
In the children's institution sample, 
a reliability score was formed based 
on the number of instances of con- 
sistent responses to 17 repeated yes- 
no items divided by the number of 
item pairs answered appropriately. 
This index was calculated only if 
sxibjects had been able to answer 
half or more of the 17 repeated item 
pairs. A comparable index was formed 
for institutionalized adults based 
on 29 yes-no items. Then, in each 
sample, this reliability score, which 
could theoretically range from 0% to 
100%, was correlated with IQ, age^ 
sex, responsiveness to the entire 
Form A interview schedule, and an 
acquiescence index, on which subjects 
were given a score of 1 as opposed 
to 0 if they said "yes" to both "Are 
you usually happy?" and "Are you usu- 
ally sad?" we expected that sub- 
jects with higher IQs and higher 
responsiveness scores indicative of 
verbal skill would be more likely 
-than lower IQ ;and less responsive 
subjects to give consistent responses 
on two occasions. At the same time, 
we predicted that sxibjects who tended 
to acquiesce might be likely, on yes- 
no items at least, to achieve high 
reliability scores simply by saying 
"yes" a great deal. No specific 
predictions were formulated about 
the relationship between age and sex 
and reliability of response. 

Reliability scores ii^' both saraples 
were normally distributed. In the 
children's sample, these scores 
ranged from 59% to ioo% "(mean = 85.8%, 
SD = 11.1%) for the 44 of 52 subjects 
for whom they were calculated. Simi- 
larly, in the adult sample, 40 of 58 



subjects had enough data to calculate 
scores, and scores ranged from 58% to 
97% (mean = 76.1%, =-10.2%). Inter- 
estingly, then, no one was inconsist- 
ent on more than half of the, items 
for which first and second adminis- 



tration answers were 



available. 



Table 6.2 presents the sln^^le corre- 
lations between indiviuual character- 
istics and the relialDility of 
response score. These correlations 
are very difficult jto interpret, 
particularly since /the findings in 
the two samples do/not correspond 
well. In the children's institution 
sample, contrary to expectation, 
lowv^r IQ subjects/ were actually more 
likely to give reliable responses 
than higher IQ subjects. Other cor- 
relations were small and inconsistent 
in direction, but there was at least 
a tendency for^ more acquiescent sub- 
jects to display more reliability of 
response. In' the adult institution 
sample, by cpntra. t, there were small 
positive associations between both 
IQ and responsiveness and reliability, 
the relatibnship between responsive- 



ness and reliability being statis- 
tically significant. In this sample, 
there was essentially no relation- 
ship between acquiescence and relia- 
bility. About the only consistency 
across samples was a very slight 
tendency for females to provide a 
greater proportion of reliable 
responses than males did. 

These simple correlations do not • 
take into account the fact that some 
of the individual characteristics 
involved in the analysis are inter- 
related. As noted in Chapter 5, IQ 
and responsiveness were positively 
associated in both samples. More- 
over, there is a consistent tendency 
for higher IQ persons to be less 
likely to acquiesce as indicated by 
responding "yes" to both happiness 
items. The correlation between IQ 
and acquiescence was .40 in the 
children''s sample, and .31 in the 
adult sample. Therefore, partial 
correlations, which allow for exam- 
ining a relationship between two 
variables, with the effect of a third 
controlled, were conducted to attempt 



Table 6.2: Individual Correlates of Reliability of Response 



/ 


Child Institution 
Reliability 


Adult Institution 
Reliability 




IQ 


-,38** 


.12 




Age 


.06 


.22 




Sex 


.12 


.19 




Responsiveness 


-.15 


.32* 




Acquiescence 


.16 


-.10 





*Sigr*:ficant at the .05 level 
♦★Significant at the .01 level 
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to unravel the independent impacts 
of IQ, acquiescence, and responsive- 
ness on the reliability measure. 
However, these analyses did not clar- 
ify the findiiags. They left the 
pattern of correlations in Table 6.2 

essentially unchanged. 

»> 

Thus, we must conclude that the 
extent to which mentally retarded 
persons give consistent answers on 
two occasions is essentially unpre- 
dictable based on the information 
we had available. It is still pos- 
sible that opposing forces were at 
work; that, on the one hand, higher 
ability subjects were likely to be 
consistent in their responses but 
that, on the other hand, acquiescent 
tendencies inflated consistency 
scores for lower ability subjects. 
Unfortunately there were not enough 
items in formats other than yes-no 
to allow us to see if reliability of 
response where acquiescence does not 
operate is more predictable. 

Summary and Conclusions 

• This chapter has been concerned 
with one aspect of the issue of 
whether ansv/ers given by mentally 
retarded persons can be considered 
useful information. In both insti- 
tution samples, repetitions of ques- 
tions a week apart provided evidence 
of response reliability. In both 
saiuples, it was found that answers 
given on two occasions were generally 
consistent over 80% of the time. 
Figures for yes-no, either-or, and 
open-ended questions with response 
categories were generally very simi- 
lar and high. The lowest consi'^^ten- 
cy figures were obtained for the mul- 
tiple choice questions used in the 
children' s institution sample, es- 
pecially those questions about sat- 
isfaction which asked respondents to 
choose among happy and sad faces. 
At the same time, as kappa statistics 
associated with these items suggested, 
it Is more difficult tp obtain con- 



sistency by chance on four-option 
questions. Thus, although the con- 
sistency figures for these items were 
relatively low, it appeared that re- 
spondents were not just answering in 
a random way. 

While we might be more gratified by 
consistency closer to 100%, we must 
conclude that what mentally retarded 
persons say one week is likely to be 
quite similar to what they say the 
next week. Most of our questions 
were factual in nature so such con-, 
sistency should be expected. In- 
terestingly, however, an analysis in 
the adult institution sample of fac- 
tual versus subjective yes-no ques- 
tions did not reveal lower reliabil- 
ity of responses to subjective ques- 
tions, despite the fact that such Re- 
sponses might be predicted to be more 
mood-dependent and changeable* from 
week to week. 

As we have noted throughout this chap- 
ter, however, reliability of response 
figures was very difficult to. 
interpret. In response to yes- no 
questions,' an unthinking endorsement 
of items would be scored as consist- 
ency of responses in.- exactly the 
same way as a thoughtful provision 
of "yes" responses on both occasions. 
Given our findings of .high rates of 
acquiescence to yes-no questions m 
retarded samples, as well as the 
positive association between jthe 
percentage of the sample endorsing 
an item and the consistency figure / 
for that item in the present analy-/ 
sis, we must conclude that relia- / 
bility figures for yes-no questions 
are inflated. At the-^very least,/ 
we must be cautious hot to construe 
them as evidence that answers given 
a week apart express, a consistent- 
body of "information" about the /live 5 
of the respondents. We can conclude 
that the picture of the group, and 
of the individual within it,,: t)b- 
tained from yes-no questions does 
not change much from week to week, 
as long as we recognize that these 
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"pictures'' may not be faithful repre- 
sentations of reality. 

Much the same can be said of relia- 
bility figures for open-ended ques- 
tions. We suspect that answers to 
them underrepresent the extent to 
which mentally retarded people are, 
for example, involved in various 
activities, in the same "way that 
yes~no questions probably overrepre- 
sent extents of involvement. While 
we did not have a completely satis- 
factory way of analyzing consistency 
for these questions, it was apparent 
that few responses were given to 
such questions and consistent men- 
tion of items was a rare phenomenon 
simply because mention of items was 
a rare phenomenon. Unfortunately, 
the reliability of such items was 
heavily influenced by the number of 
categories that the coders chose to 
construct. The fact that reliability 
for categories with frequent mentions 
tended to be lower than reliability 
for categories with virtually no 
mentions is not a healthy sign. 

AS for items with structured choices, 
they were associated with relatively 
low reliability figures. While 
either-or items fared well despite 
the fact that they were subjective in 
nature, verbal multiple choice items 
calling for an indication of extent 
of activity were only marginally 
reliable. Pictorial multiple choice 
questions about satisfaction with 
aspects of institutional life were 



not responded to i^ he Scur. way on 
two occasions, even though obtained 
consistency figures exceeded chance 
probabilities. These figures, too, 
are difficult to interpret. Subjec- 
tive feelings about the institution 
might well change from week to week 
as a function of mood and intervening 
experience, even "true" answers- are given 
given on both occasions. 

All in all, then, our analysis of 
reliability is encouraging in the 
sense that answers can generally be 
counted on to be the sairie from week 
to week, but discouraging in the 
sense that we are not sure how to 
interpret even high reliability 
figures. Moreover, we were unsuc- 
cessful in identifying which persons 
are most likely to provide reliable 
data. This uncertainty will, however, 
be reduced as we turn to issues of 
response validity. In the next chap- 
ter, we will take up the question of 
whether clients' responses agree with 
responses to the same questions pro- 
vided by attendants and parents. In 
this analysis, we will be able to 
look more directly for evidence of 
systematic biases in the responses 
given by mentally retarded persons. 
Such biases as acquiescence, selec- 
tion of the, -last option on either-or 
or multiple choice questions, and 
failure to mention things mentioned 
by significant others, to the extent 
that they occur^may cast the reli- 
ability figures presented here in a 
different light. 
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Chapter 7 

TO WHAT EXTENT ARE ANSWERS VALID? 



The two previous chapters hav6 
dealt with responsiveness of men- 
tally retarded persons to interview 
questions and with the reliability 
of their responses. While it is 
clear that these factors are im- 
portant, it is equally clear that 
even reliable responses to questions 
are useless if they are not valid. 
That is, it is pointless to elicit 
responses which do not accurately 
reflect the attitudes and behaviors 
of the mentally retarded persons 
interviewed. We have devoted con- 
siderable attention to evaluating 
the validity of responses by our 
mentally retarded respondents. > 

Our main- strategy for assessing / 
validity has been to compare thfe / 
responses - of^ clients with the" re - 
sponses of adults who should be / 
knowledgeable about the mentally 
retarded interviewees. Thus, when 
we asked a child "Do you go to 
church?"' we asked the child's/parent 
or cottage attendant if the child 
goes to church and noted the per- 



centage of cases in which the two 
answers agreed. Clearly, there are 
instances in which the parent or 
attendant will not be in a good 
position to provide information re- 
garding the mentally retarded respond- 
ent. Especially when the attitudes 
of retarded persons are at issue, 
attendants' or parents' opinions may 
not be accurate or even meaningful - 
Then, too, we must recognize that 
even on points of fact parents, and 
perhaps especially attendants, may 
/simply not be knowledgeable, and 
/their cinswers, like those of re- 
/ tarded persons, may be biased. None- 
/ theless, we assxime that the degree 
^ of correspondence of the responses 
. of children' and their significant 
^others is a useful (although admit- 
tedly imperfect) indicator of re- 
sponse validity, 

A second strategy for evaluation of 
response validity has been to ask 
questions to which the correct answer 
is known. This technique reduces 
the ambiguity inherent in the first 
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strategy. If, in response to such 
questions, mentally retarded persons 
are able to provide accurate infor- 
mation, we have sound evidence of \ 
their ability to give accurate re- 
sponses to at least some interview 
questions. 

Our use of these techniques in the 
course of this project was extensive. 
Therefore, a discussion in detail 
of all validity data would be im- 
possible. Fortunately, the analyses 
of agreement in the different sam- 
ples yielded ^trikingly consistent 
results. Thus, we will discuss in 
detail the comparisons of children's 
samples, using that data to illus- 
trate the major conclusions. Sxib- 
sequently, agreement data from the 
additional samples will be discussed, 
citing only those data which either 
contradict the conclusions derived 
from the community children's sample 
or which add novel insights. Then, 
the validity data derived from the 
use of questions to which answers 
* are known will be discussed. 
Finally, we will look for differences 
among mentally retarded persons in 
their tendencies to give valid re- 
sponses as indicated by agreement 
with significant others. 

Community Children , 

OVERALL AGREEMENT ' 

Sixty-one questions were aske!^ 
of both community children and th^ir 
parents, and the degree of corre- 
spondence between their response?' 
was evaluated as a check of response 
validity. Average agreement for 
this set of questions was 64.5%; 
that is, parents and their retarded 
children were in agreement about two- 
thirds of the time. Overall agree- 
ment and a breakdown of agreement 
according to question type are de- 
picted in Table 7.1. Clearly 
agreement did vary as a function of 
question type, but as the discrepant 
figures for two different types of 



multiple choice questions suggest, 
it is necessary to look more closely 
at both question format and question 
content in order to understand what 
makes agreement likely to be high or 
low. 

YES-NO QUESTIONS 

Yes-No Questions With Words Only 

Average Agreement 

Questions posed of non- 
institutionalized children and 
their parents included 22 simple 
verbal questions, most about client 
activities, requiring "yes" or "no" 
responses. The responses of clients 
and their parents were compared; data 
for these 22 comparisons are summa- 
rized in a mean contingency table 
(Table 7.2) . 

Children and parents tended to agree 
in their responses to yes-no ques- 
tions; on the average, 65.7% of 
comparisons yielded child-parent 
agreement. As can be seen, agree- 
ment tended to result 'primarily 
because both clients and parents 
responded "yes." Because clients 
favored "yes" slightly more than did 
their parents, the majority of. dis- 
agreements occurred when the client 
said "yes," and the parents said 
"no," suggesting that acquiescence 
by retarded persons may threaten the 
validity of their answers. 

Differences in Agreement Due to 
Question Topic 

The use of averaged data can ob- 
scure important factors underlying 
responding. In fact, agreement 
varied greatly from question to 
question, ranging from 92.8% ("Do 
you watch TV?") to 40.8% ("When 
your 're not at school, are you usually 
by yourself?"). In pursuit of ex- 
planations for this variation, we 
examined in some detail the differ- 
ences in agreement rates within 
this group of questions. First, 
it would appear that relatively 
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Table 7.1: Agreement by Question Type 2 



Question type 


/^"P ^^^^^ a.c^ 1 <^"n C 
TT on ^UOO UX^Jilss 






Yes-no (words only) 








les— no vWXun ±eau— xn tn pictureb; 


in 






Either— or (subjective; words only) 


7 








4 


58.1 




Mt^stiple choice (discrete) 




77 Q 




Multiple choice (quantitative) 


4 


24.0 




Open-ended (factual) 


64.8 




* Open-ended (coded responses) 


3 


77.9 












Total 


61 


64.5 



♦Calculated by averaging the average agreement scor^^^L across response categories 
for the three questions 



simple and concrete questions yield 
higher agreement rates than do more 
complex and abstract questions. 
For example, agreement for "Do you 



go to church?" was 78.2%, as opposed 
to 56.4% for "Most of the time, 
is it up to you to decide what time 
you go to bed?" 



Table 7.2: Mean Contingency Table for 22 Verbal Yes-No Questions 



Responses of Clients 


Responses 


of Significant Others 








No i 


Yes 






No 


13o0% 




' 14ol% 


27ol% 




Yes 


20ol% 




52.7% 


72o8% 




Total 


33ol% 


66o8% 







Agreement = 65 o 7% 
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Five questions which asked about 
the client's participation in various 
activities had an average agreement 
rate of 82.3%, in contrast to 66.5% 
for a group of seven more complex 
and abstract ■ questions dealing with 
training in specific skills ("Is 
anybody at school teaching you about 
money npw?""i's an example of the 
latter group of questions) . 

Second^ it appears that some topics 
include built-in response biases 
which influence agreement. For 
example, three questions which in- 
quired about client's possession of 
particular skills ("Do you know how 
to read books?") resulted in agree- 
ment lower than that »f or any other 
topic area — 54.3%. It seems un- 
likely that this is attributable to 
the difficulty of these questions'; 
rather, a specific bias appears to 
be in operation here. That is, 
clients frequently claim skills, 
which their parents deny they possess. 
On the average, 81.1% of clients 
and only 52.7% of parents answered 
"yes" to the skills questions, sp 
that fully 37.0% of response com- 
parisons were disagreements in which 
clients claimed skills and were, 
contradicted by their parents. 

Is this a matter of client's overes- 
timating their skills or parents 
landerestimating the skill levels of 
their mentally retarded children? 
Although we cannot be certain, the 
former possibility seems more likely 
in. this case. In responding to 
skills questions, clients probably 
become concerned with f avor.able 
5elf-presentation so that a desire 
to appear comppcent encourages 
an affirmative answer. VVes" answers 
probably also reflect a more general 
tendency ' '^•' .\rd acquiescence, but 
concern wi."" . -presentation is 
probably wha\. -nts for tlrie par- 

ticularly low agreem^mt on these 
, skills questions- 



Differences in 'agreement as a func- 
tion of parents' responses 

Another pattern appeared in 
variations of agreement rate within 
the 22 yes-no questions. Specifi- 
cally, agreement was highest for 
those questions to which the parents 
are most inclined to respond "yes." 
This is to be anticipated in light 
of, an observation made throughout 
the course of this project that men- 
tally retarded respondents tend to 
acquiesce, or to respond "yes" re- 
gardless of question content. To the 
extent that acquiescence on the part 
of the clients occurs in response 
to yes-no questions, higher agreement 
should be observed on those questions 
for which the response of significant 
others is "yes." Here agreement 
would not indicate response validity 
as much as a happy coincidence of 
response bias and reality. 

To test that possibility, this group 
of questions was divided into the 11 
questions which parents tended most 
often to answer "ye?" (average of . 
88.9% "yes" responses by parents) and 
the 11 questions to which parents 
were least inclined to respond "yes" 
(average 44.6%). Comparison of the 
agreement' rates for these two groups 
of questions is depicted in Table 
7.3". As can be seen, average 
agreement for "high yes" questions 
was 75.7%, as opposed to 55.6% for. 
the "low yes" questions. Looked at 
another way, the percentage of 
parents responding "yes" to these 
22 questions correlated, .80 with 
agreenvent figures for the questions. 
This comparison suggests that clients 
tend to indiscriminately respond 
"yes" to yes-no questions, and that 
client-^parent agreement is to some 
extent artificially controlled by 
the proportion of "yes" responses 
given by parents. Thus overall 
agreement rate provides an inflated 
estimate of validity when the ques- 
tions involved are predominantly 
answered '*'yes" by significant others. 
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Table 7.3: Child-Parent Agreement as a Function of Parent's Tendency 
to Say "Yes" 



11 Questions, Low Proportion of Parent "Yes" Responses 



Client 


significant Other 






No 


Yes 


Total 


'i 

No 


22o6% . 

/ 


llo6% 


34o2% 


Yes 


32o7% 


33o0% 


65.7% 


f 


55o3% 


44o6% 





/ 



Agreement = 55.6% 



11 Questions/ High Proportion of Parent "Yes" Responses 



J 

client 


Significant Other 




No 


Yes 




No 


3.3% 


16.5% 


19.8% 


Yes 


7o6% 


72.4% 


80.0% 




10 o 9% 


88o9% 





Agreement = 75 o 7% 



A set of questions tp which 50% of 
parents' responses were "yes" would 
provide a better test of validity of 
yes-no ' questions . 

Conclusions 

In short, these data suggest 
three conclusions. First, cogni- 
tively demanding questions posed of 



mentally retarded individuals tend 
to elicit less valid responses than 
do easier questions (to' the extent 
that comparison with their parents' 
responses constitutes cin adequate 
measure of validity). Second, spe- 
cific topic areas (for example, ques- 
tions about skills and competencies) 
may involve biases which tend to 
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systematically alter responses in 
Such a way as to seriously reduce 
validity. Third, average agreement 
rate is an overestimate of validity 
for this group of questions, for 
which the parents' response was "y^s" 
66 .8% of the time. Where parents 
respond "yes" with frequency, acqui- 
escence on the part of retarded re- 
spondents inflates agreement figures 
but not necessarily validity. It is 
likely, in light of this observation, 
that the actual validity of these 
yes-no questions is low enough that 
their usefulness as a means of 
gathering information from mentally 
retarded persons is questionable. 

Yes-No Questions 

with Lead-in and Pictures 

Interviews with community chil- 
dren included 10 yes-no questions 
.which were preceded by a verbal 
lead-in and which used pictures to 
clarify questions. These 10 ques- 
tions concerned chores, and the ver- 
bal lead-in read: "Here are some 
pictures of different kinds of chores 
some people do. I want you to tell 
me if you do any of these chores at 



home." The questions were simple 
yes-no questions such as "Do you set 
the table?" and "Do you make beds?" 
each question being accompanied by 
a picture of youngsters engaging in 
the chore involved in the questions. 

Average agreement 

Data for the 10 questions were 
averaged and ar^ depicted in a mean 
contingency table (Table 7.4). 
Averr.ge agreement for these questions 
was 71.5%. Nearly two-thirds of the 
response comparisons were agreements 
in which both child and parent re- 
sponded "yes." Inconsistencies were 
divided between cases in which the ^ 
-Client responded "yes" and the parent 
"no," and the converse; thus, there 
was not strong evidence of acquies- 
cence for the set as a whole. In 
short, agreement for this class of 
questions was relatively high, some- 
what higher than for the verbal yes- 
no questions, and tended to result 
because both client and parent re- 1 
sponded "yes" to most questions. 



Table 7.4: Mean Contingency Table for 10 Yes-No 
Questions with Lead-in and Pictures 



Responses of Clients 


Responses of Significant Others 


No 


No 


Yes 


TOTAL 




6o9% 
13o7% 


14o7% 
64o6% 


21„6% 
78.3% 


Yes 


' 20o6% 


79.3% 





Agreement = 7l„5% 
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Differences in agreement 

There was relatively little 
variability in agreement among these 
questions. For 9 of the 10, the 
ran^e of agreement was only from 
65.5% to 85.5%. It may be argued 
that the uniformly high agreement for 
these questions results from the fact 
that the "valid" answer to most of 
them (as reflected by the parents' 
response) was "yes." However, agree- 
ment for "DO you cook on the stove?" 
was only 47.2%, by far the lowest 
agreement for the set. It is not 
coincidental that this question also 
had the lowest percentage of "yes" 
responses from parents (52.7%) . 
Clients tended to respond "yes" to - 
this question (65.4%), with the 
result that 32.7% of the comparisons 
were inconsistencies in v-liich clients 
responded "yes" while their parents 
were responding "no." This inter- 
pretation is further supported by a 
comparison with the question from 
this group which had the highest pro- 
portion of "yes" responses from 
parents: "Do you pick up stuff 
around the house?" Ninety-six per- 
cent of parents answered "yes," and 
the agreement.. rate was 85.5%, highest 
for this set of 10 questions. All 
agreements in this case resulted be- 
cause both client and parent respond- 
ed "yes." For this group of ques- 
tions, the correlation between pro- 
portion of parents responding "yes" 
to a question and agreement for that 
question was .97. 

Conclusions 

The high average agreement for 
these yes-no questions appears to 
overestimate response validity. 
Average agreement is high because 
the apparently appropriate answer to 
most of these "chores" questions is 
"yes." When parents give a high 
proportion of "no" responses, client- 
parent agreement drops accordingly r 
as it did on verbal yes-no questions. 



EITHER-OR QUESTIONS 

Either-or Questions with W ords Only 
Seven questions asked of commu- 
nity children and their parents were 
simple verbal questions posing two 
response alternatives. "Are you 
usually happy?" for instance, is the 
complement of the above example. 

Average Agreement 

Agreement for this group of ques- 
tions ranged from 50.0% to 86.0%; 
average agreement was 65.6%. Average 
agreement was thus comparable to that 
for yes-no questions using words only 
(65.7%), but' the range for those yes- 
no questions was much greater (40.8% 
to 98.2%). To what may this decreased,^ 
variability for the either-or ques- 
tions be attributed? As has been 
pointed out, the upper end of the 
distribution of agreement rates for 
yes-no questions appears to be an 
overstatement of the validity of 
those questions, ,due t^o acquiescence 
by mentally retarded respondents and 
the fact that the pa^rent's response 
to those questions is typically "yes." 
It seems likely that* agreement rates 
for either-or questions are more uni- 
form because no such biases operate 
in these\ questions to generate some 
spuriously high estimates of validity. 
- Indeed, we ^ have found no reason to 
suspect that agreement rates for 
either-or questions are anything 
other than an accurate indication of 
the validity of responses to these 
questions, for disagreements were 
generally mixed in nature. Despite 
roughly equivalent agreement rates 
for the two types of questions, then, 
it 'seems likely that either-or ques- 
tions generate responses of greater 
validity. Agreement for yes-no 
questions is inflated simply because | 
our survey unfortunately included a 
predominance of questions. which were | 
commonly answered "yes" rather than "r 



Differences in agreement 

No difference in agreement due to 
the reversal of order of the response 
alternatives was apparent. The rates 
of agreement for two of the three 
pairs did not differ (50.0% vs. 51.0% 
in one pair, and 72.5% vs. 73.2% in 
the other). In the third pair, the 
"happy or sad" example stated above r 
agreement figures for the two forms, 

were 86,0% (with "happy" first) and 
72.0% (with "sad" first). No expla- 
nation for this difference is readily 
apparent. Where we have observed 
order effects on either-or questions 
in our interviews, they have typi- 
cally taken the form of endorsement 
of the last-mentioned alternative 
by mentally retarded respondents. 
That bias, .however, does not account 
for the agreement difference observed 
in this pair of questions. In fact, 
more clients said they were usually 
happy when "sad" was the last men- 
tioned alternative. It would appear 
that agreement for these either-or 
questions does not differ systemati- 
cally as a function of the order in 
which alternatives are presented. 

There do, however, appear to be dif- 
ferences in agreement due to question 
topic, just as there were for yes-no 
questions. While it is difficult to 
predict which topics will yield 
agreement with significant others, 
variability in agreement as a func- ^ 
tion of question topic at least 
suggests the need for caution in 
interpreting data- gathered in inter- 
views with mentally retarded persons, 
and also suggests that we be conserv- 
ative in generalizing demonstrations 
of validity across content areas. 

! 

Either-or Questions With Pictures 

The group of questions asked of 
community children and their parents 
included two pairs of either-or ques- 
tions which used pictures as the 
response alternatives. The members 
of each pair involved the same con- 



tent but reversed the order in which 
the response alternatives were pre- 
sented. For example: "This picture 
(point) shows a boy/girl who is 
happy, and this one (point) shows a 
boy/girl who is sad. Which picture 
shows how you usually feel? Point 
to the picture. " 

Average agreement for these either- 
or questions with pictures .was 58 . 1%. 
Agreement for the individual questions 
ranged from 41.5% to 74.1%. Again, 
it appears that agreement yaries more 
or less unpredictably across topic 
areas. Agreement was much higher for 
the two happy-sad questions (74.1% 
and 66.7%) than for the other two 
qu/stions, which asked whether the 
client spends free time alone or with 
otniirs (41.5% and 50.0%). Disagree- 
ment on the latter pair of questions 
tended to result because oliehts 
chose the "alone" pictures, while 
parents responded "with others." The 
agreement rate for these four ques- 
tions did not vary systematically as 
a function of the order in which 
response alternatives were presented. 
Moreover, the* use of pictures as 
response alternatives did not improve 
agreement over that for verbal either- 
or questions on the same topics. If 
any difference can be said to exist, 
it is that either-or questions with 
words yield somewhat superior agree- 
ment .(64.8% vs. 58.1%). This appar- 
ent decrease in validity for picture 
question&^^^ay be. due to the fact that 
the use of pictures increases respon- 
. siveness. Many individuals who do 
not respond verbally can respond by 
pointing to pictures. The response 
patterns of these less verbal, lower 
IQ respondents may be largely random, 
reducing validity. We do in' fact 
have some evidence that the responses 
of lower IQ respondents tend to'be 
less valid (see discussion at end 
of 'chapter) . 
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conclusions 

Average agreement for either-or 
questions is similar to agreement for 
yes-no questions. Because agreement 
figures probably give an overestimate 
of the validity of yes-no questions,, 
we can conclude that either-or ques- 
tions generally yield more valid 
responses. There is no evidence 
suggesting that agreement is not an 
accurate indicator of validity for 
either-or questions. Agreement does 
not appear to be affected by revers- 
ing the order of presentation of re- 
sponse alternatives, although it 
does vary as a function of question 
topic. Finally, the use of pictures 
as response alternatives apparently 
decreases validity of responses to 
either-or question^j^omewhat , prob- 
ably becauoe less capable, clients 
can then provide answers. 

MULTIPLE CHOICE QUESTIONS 

Eleven questions posed of non- 
institutionalized children and their 
parents were multiple choice ques- 
tions, involving a choice between 
either three or four response alter- 
natives, ^verage agreement was 57.0% 
These questions differed in question 
format: some were three-choice and 
some four-choice questions, some 
used words only and some words plus 
pictures in depicting response alter- 
natives/ and some of the questions 
were follow-ups to yes-no questions. 
However, there wer6 no systematic 
differences in agreement due to any 
of these format considerations. 

Agreement for multiple choice ques- 
tions did, however, differ according 
to the type of response alternatives . 
Six multiple choice questions gave 
discrete alternatives- about matters 
of fact*. ("Do you live in ,a house, an 
apartment building, a trailer house , 
or a duplex?") , while five used 
quantitative dimensions as response 
alternatives ("How many friends do 
you have: a lot, some, not^many, or 



none?'*^o Average agreement for the 
discrete questions was 77.9% (range 
70.5% to 85.5%) f average agreement 
for the quantitative multiple choice 
questions wac3 only 24.0% (range 25.9% 
to 39.6%). AS is readily apparent, 
not only does average agreement for 
the tvro groups of questions differ, 
but their re.nges do not come close 
to overlapping. Clearly, agreement 
was superior for the questions 
offering discrete alternatives. Be- 
cause quantitative responses require 
finer discriminations, this trend 
might well be present in normal pop- 
ulations as well. However, the fact 
regains that average agreement was 
vety high for the discrete questions 
and unacceptably low for those 
offering quantitative alternatives. 
This suggests that discrete multiple 
choice questions may be useful in 
interviewing mentally retarded per- 
/sons, but it would appear pointless 
/to ask mentally retarded respondents 
/ to make quantitative judgments, 
/ given the absence of evidence for 
■ validity of their responses. 

^ / Conclusions 

*l Multiple choice questions . 

offering discrete response alterna- 
tives yield very high client-parent 
agreement* Agreement for questions 
using cruantitative alternatives is 
so low as to make these questions 
useless in obtaining information 
from mentally retarded respondents. 
No conclusion can be offered re- 
garding format differences such as 
number jf options and -method of 
presertation of response alternatives 

OPEN-ENDED QUESTIONS 

■' ^> 
Open-ended Factual Questions 

Questions allowing comparisonib 
of the responses of children and 
their parentis included four open- 
ended questions having a single 
correct answer. ' On the average, 
children matdhed their parents 64.8% 
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of the time. It is somewhat dis- 
couraging that validity was this low 
for . simple questions requesting fac- 
tual information which should be 
readily available to mentally re- 
tarded respondents. Agreement was 
only 42.5% foir the question "Counting 
you, how many people live in your 
house righr now?" The other three 
questions asked the client's age, 
the starting time for school, and 
how the child gets to school; Agree- 
ment figures for these questions 
were 75.0%, 56.3%, and 85.4%, re- 
spectively. In short, it appears 
that open-ended questions requesting . 
factual information yield responses 
of varying validity, depending on 
the questions. Apparently ques- 
tions involving nixmber and time 
concepts were difficult. The dis- 
couragingly low agreement on some 
of these questions encourages a 
conservative attitude when inter- 
preting data drawn from interviews 
with mentally retarded respondents. 



O pen-Ended Questions with Coded 
Response Categories 

Three open-ended questions posed 
of community children and their parents 
required subsequent coding of responses 
into categories. For example, responses 
to the questions "What things would 
you really like to learn in school?" 
were assigned to such categories as 
academics, physical recreation, and 
self-care skills. For each of these 
questions, ten such response cate- 
gories were defined. Comparisons were 
conducted on a category-by-category 
basis / so that each category was coded 
either "mentioned" or "not .mentioned" 
by client and parent* Responses were 
then compared in a 2X2 contingency 
table, computed by averaging across 
all 30 response categories (Table 7.5). 
Average agreement of these 30 categories 
was 77.9%. However, as can be seen, 
most of this .agreement resulted because 
"both child and parent did not men- 
tion a category. Only 6.8% of , 
client-parent compsirisons yielded 
agreement because both client and 
parent mentioned a particular 
category. It is difficult to drav; 
conclusions from these data, because 
agreement depends to a large extent 



Table 7.5: Mean Contingency Table for 30 Coded 

Response Categories to 3 Open-ended Questions 



Responses of clients 


Response of Significaiit Others 




1 

Did not mention 


Mentioned 






Did not mention 


. 71.1% 


10.7% 


81.8% 




Mentioned 


11.4% 


6.8% 


18.2% 




Total 


82.5% 


17.5% 







Agreement = 77.9% 
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on the verbosity of respondents and 
the number of . categories developed 
by the coders, it is at loast clear, 
that these open-ended, questions 
yielded very little information. 
It is also instructive to note t^at 
the four response categories which 
yielded the highest porportion of 
mention-mention agreeinents were the 
four categories lowest in overall 
child-parent agreement. This sug- 
gests' that when clients and their 
significant others do offer infor- 
mation in response to open-ended 
questions, the responses they make 
frequently differ. In view of this 
lack of evidence for the validity of 
responses to open-ended questions, 
as well as the low responsiveness 
rates for these questions, it 
would appear that this format holds 
little promise in interviewing men- 
tally retarded persons. 

Conclusions 

Analysis of the open-ended ques- 
tions suggests two major conclusions. 
First, information drawn -from open- 
ended questions must be evaluated 
with caution. The low . agreement 
obtained on simple factual questions 
and the ].arge variability in agree- 
ment across questions support this 
observation. Second, the open-ended 
question format is a relatively un- 
attractive one. These questions 
tend to pull very little information 
from mentally- retarded persons, and 
the information obtained from. open- 
ended questions has not been vali- 
dated by agreement with parents' 
responses. 

CONCLUSIONS 

^ General Considerations 

These' data regarding the validity 
of the responses of mentally retarded 
persons are not encouraging. It is 
clear that information drawn from 
interviews with retarded persons 
must be evaluated critically. 



Children's responses frequently fail, 
to correspond closely to the re- 
sponses of their parents. It is 
particularly notable that this lack 
of agreement exists for some simple 
questions regarding factual infor- 
matiqn which should be readily 
available to most persons. Second, 
it is apparent that different 
questions yield responses of dif- 
ferent validity. In some cases, 
particular topics seem to be af- 
fected by systematic response biases- 
Furthermore, conceptually more 
difficult questions appear to yield 
less valid responses. However, it 
seems difficult to predict in ad- 
vance which questions will yield 
very low client-parent agreement. 
This fact is of vital importance 
because it implies that response 
validity can never be globally 
assumed; rather, each question must 
be interpreted conservatively since 
any individual question may be es- 
pecially subject to response effects. 
Unpredictable variability, coupled 
with the very low validity of many 
responses/ requires that researchers 
who interview mentally retarded 
persons design interviews which deal 
very carefully with the issue of 
validity. 

. , , ■ / 

Format Recommendations / 

The fact that major differences 
exist in client-parent agreement as 
a function of question format sug- 
gests that some formats are pref- 
erable to others. Yes-no questions 
appear especially unsuitable due to 
the effects of acquiescence. 
, Client-parent agreement frequently 
drops to below chance levels when 
a high proportion of parents re- 
spond "no" to these questions. 
Either-or questions, on the other 
hand, appear to have particular 
promise. Agreement for these ques- 
tions was relatively high and 
appeared to 'be uninfluenced by 
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systematic biases. Multiple choice 
questions offering response alter- 
natives which differ quant itat:.vely 
are very \anattractive; agreement on 
these questions was below chance 
level in this sample. In contrast* 
multiple choice questions offering 
discrete response alternatives and 
dealing with basic factual informa- 
tion yielded the highest agreement 
rates of any format. Further ex- 
ploration of the potentials of dis- 
crete multiple choice questions 
seems appropriate. On the basis 
of client-parent agreement the open- - 
ended question forrat appears un- 
desirable. Average agreement was 
reasonably high when the question 
called for a single factual answer, 
but open-^.ended questions calling 
for an active listing of responses 
are probably useless. Moreover, 
the fact that many retarded persons 
cannot answer open-ended questions 
means that even when they yielded 
valid ans^f'cr;-; those answers come 
from only a limited segment of the 
pdpulation. In short, either-or 
questions or multip3.t: choice ques- 
tions offering di;-cuete alternatives 
would appear to have the most promise 
as methods of obtaining information 
fro* ..lentaliy regarded respondents. 

ANALYSIS 01- JiGR^RMENT USING 
COEFFICIENT IO\PPA 

In the preceding discussion of 
agreement in the community children's 
sample, little mention was made of 
any standard against which agreement 
figures can be compared. We have 
simply compared agreement figures 
and applied an intuitive standard 
based on the anticipated ranges of 
agreement figures. At one extreme, 
if responding were completely random 
on the part of bpth clients and sig- 
nificant others, agreement for ques- 
tions with two response alternatives 
would average 50%. At the other 
extreme, perfect agreement between 
clients and their sianificant othe3?s 

i 



would result in agreement of 100%. 
An agreement figure can be roughly 
evaluated according to where it 
falls on this continue. Agreement 
lower than or little better than 
50% suggests low validity, while 
agreement approaching 100% i^ im- 
pressive. (One must, of course, 
remember that agreement is a rough 
estimate of validity; e.g., for 
some questions, acquiescence in- 
flates agreement but actually raises 
questions about response validity.) 
Using this intuitive standard, the 
82.0% agreement for the hypothetical 
questions depicted in Table 7.6 
appears to be reasonable evidence 
of response validity. 

However, as was done in Chapter 6 
on reliability, kappa coefficients 
■ can be calculated to compare ob- 
tained agreement to "change'* agree- 
ments as defined by reference to 
the resppnse patterns (i.e., mar- 
ginals) of the two groups. Given 
the marginals depicted in Table 7.6, 
the extent of agreement expected by 
chance for the hypothetical question 
is 82.0% (.9 X .9 + .1 X .1). That 
is, given that 90% of clients and - 
90% of significant others respond 
"yes" to this question, then 82.0% 
of. client-significant other pairs 
would agree if those pairings were 
^ established by drawing randomly 
from the two populations. Thus, the 
actual agreement figure of 82.0%, 
impressive according to the intuitive 
standard, is actually no different v. 
than the . extent of agreement to be • 
expected by chance, and the kappa 
analysis makes us wary of making too 
much of the extent of agreement 
obtained. 

If we were concerned only with the 
aggregate level of analysis, refer- 
ence to agreement figures and ^ 
associated kappas would not even be 
necessary. It would be adequate 
/ to look only at the marginals of 
/ contingency tables. If, as in the 
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Table 7.6: Hypothetical Yes-No Question 



Clients 



Significant Others 





No 


Yes 


Total 


No 


1 


.0% 


9.0% 


10.0% 


Yes 


9 


.0% 


81.0% 1 


90.0% 








1 




Total 


10 


.0% 


90.0% / 

/; 





Agreement = 82.0% 



example in Table 7.6, the percent- 
ages, saying "yes" in each group 
were identical or nearly identical, 
we would have a basis for concluding 
that the picture of the sample 
obtained is virtually the same 
whether one listens to clients or 
to their significant others. This 
type of concurrence would, of course, 
be possible even if there were nvimer- 
ous discrepancies between individual 
clients and their significant others. 

If, on the other hand, one is con- 
cerned (as we have been) with, 
validity at the individual level of 
analysis, attention tp paired 
responses is required. We have 
generally been content to use an 
intuitive standard in judging agree- 
ment figures, for we feel. that 70 
to 80% or greater agreement between 
' clients and significant others is 
impressive. Moreover, it means that 
we would have confidence, if we were 
to conduct correlational analyses 
using either client or significant 
other data in its place, that 
findings based on the two groups of 
informants would generally be 
similar. However, we can still ask 
whether a given agreement figure 
• really represents agreement beyond 



what one would obtain if client and 
significant other responses were 
paired randomly rather than correctly. 

It is for this purpose that we have 
chosen to use kappa selectively 
simply to provide an additional 
perspective on consistency and agree- 
ment by chance; it is difficult for 
kappa to be large and significant 
when^the marginals are lopsided 
simply because the probability of 
agreement by chance becomes large. 
Given an agreement figure 80 to 90%, 
we can still be satisified that . 
clients and their signif icany others 
are saying much the same things, 
even if kappa is not signifxcant. On 
the other hand, we cannot l^e fully 
. satisfied by . low agreement figures 
even if kappa is large, f^r we would 
riot be confiderit using data from 
clients and significant pthers as if 
they were interchangeable. 

We can be most satisfied if agreement 
is high and kappa is l^igh, for then 
we have reason to conclude that the 
'3xtent of agreemejnt obtained is 
greater than what would be expected- 
if responses from the two groups 
were paired randomly. 



AGREEMENT IN THE CC»1MUNITY CHILDREN'S 
SAMPLE BY THE KAPPA CRITERION 

Overall Agreement i 

The use of kappa to analyze 
agreement yields discouragjing results. 
Of 88 client-parent comparisons for 
the community children's sample, only ^ 
11 resulted in agreement which 
differed significantly from chance 
(at the .05 level) according to this 
standard • 

Agreement by Question Format 

Multiple choice questions offering 
discrete response alternatives emerge 
as the most desirable format when 
kappa is the criterion. All six of 
these multiple choice questions 
generated agreement significantly 
different from chance. To cite an 
example, for the question "Do you_ 
live in a duplex, a house, a trailer 
house, or an apartment building?" 
' agreement was 81.6%, with kappa = .44 
(£ less than .05, one-tailed). It 
is unfortunate that these six ques- 
tions all dealt with either the 
client's means of transportation to 
school or with his/her dwelling plaice, 
as these topics were not adequately 
treated in other formats. Therefore, 
the possibility cannot clearly be 
ruled out that it is these topics, 
rather than the question format, which 
engender unusually valid responses. 
In short, analysis using kappa seems 
to recommend discrete multiple choice . 
questions even more strongly than did 
the previous analysis. However, none 
of the five multiple choice questions 
•using quantitative response alter- 
natives attained significance 
according to the kappa criterion. 
Thus the kappas reinforce the con- 
clusion that these types of multiple 
choice questions are not associated 
with high validity of response. 

Of the 32 yes-fio questions in this 
sample, only, three generated agree- 
ment significantly different from 



chance according to the kappa 
criterion. All three of these 
questions were simple verbal 
questions having to do with 
teaching skills. For example, 
for the question "Is anybody at 
school teaching you about cooking 
now?" agreement was 71.1%, with 
kappa = .42 (p less than .01, 
one-tailed) . The low percentage 
of yes-no questions for which 
agreement was significant by this 
criterion highlights the conclu- 
sion based on the earlier analysis: 
yes-no questions do not elicit 
generally valid responses from 
mentally retarded respondents. 
However, it should be noted that 
the high proportions of "yes" 
response;s on many of these ques- 
tions mean that a large kappa is . 
difficult to obtain .^--^ 

The remaining two comparisons for 
which kappa .indicates agreement 
differing significantly from 
chance were open-ended questions. 
One of the four open-ended factual 
questions, "Most days, how do you 
get to school?" resulted in agree- 
ment of 85.4% (kappa = .68, p 
less than .01, one-tailed). The 
fact that the content of this 
question coincides with the content 
of three of the discrete multiple . 
choice questions may suggest that 
the especially impressive agree- 
ment for these questions is at 
least .partly attributable to ques- 
tion content. Additionally, one 
of the 30. response categories for 
open-ended questions with coded 
responses generated agreement sig- 
nificantly different from chance 
by the kappa criterion. For the 
question "Do you play any sports? 
(if yes) What sports do you play?" 
the category "basketball" generated 
agreement of 75.0% with kappa = .43 
(p less than .01, one-tailed). 
Given only two significant agree- 
ment figures for the 34 client- 
parent comparisons for the open- 
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ended questions, the Kappa analysis 
does not appear to challenge our 
earlier conclusion that open-ended 
questions are not particularly 
appropriate for interviews with 
mentally retarded respondents. 
Again, however, the high pro- 
portions of "no mention" re- 
sponses among both children and 
parents make the probability of 
agreement by chance correspondingly 
high. 

One conclusion based on the intu- 
itive analysis of agreement is 
somewhat in opposition to the 
conclusions drawn from the kappa 
analysis. That is, none of the 
eleven either-or questions in this 
sample resulted in agreement 
differing significantly from chance 
by the kappa criterion. Because 
our earlier analysis suggested that 
either-or questions are, along with 
discrete multiple choice questions, 
the most desirable question type, 
this observation is problematic. 
We can again point out that the 
marginals for either-or questions 
are lopsided, elevating the standard 
for significant agreement. We can 
^still conclude that agreement is 
substantial and that systematic 
response biases (e.g., selection 
of^the last option) are relatively 
weak, but we are forced to be more 
tentative about the merits of 
either-or questions. 

In jthe analyses of agreement in the 
remaining samples which follow, we 
will not present kappa statistics. 
Ouri discussions will emphasize the 
sheer degrees of agreement '"obtained 
as Well as the. systematic biases 
accounting for client-significant 
other disagreement. Conclusions 
based on kappa statistics are 
generally parallel from sample to 
sample. Repeatedly we find that 
simple questions yield similar re- 



sponse breakdowns, often lopsided 
ones, among both clients and signif- 
icant others. Whether or not agree- 
ment exceeds chance expectation is 
perhaps less important than whether 
or not clients and significant 
others are in substantial agreement. 

Additional Samples 

Extensive comparisons were made 
of the responses of clients and 
attendants in both the institution- 
alized adult and institutionalized 
children samples. The vast majority 
of this information echoed -the con- 
clusions stated above. The following 
discussion will examine those in- 
stances in which data from the in- 
stitutionalized children or adults 
either contradicted or supplemented 
the conclusions drawn from the com- 
munity children sample. 

Overall Agreement 

A total ot 126 questions per- 
mitted comparison of responses given 
by institutionalized adults with 
responses given by their attendamtS; 
56 questions were asked of both in- 
stitutitionalized children and^their 
attendants. Average agreement 
figures for thes6 samples and for 
the community ^.^children are compared 
in Table 7.7. Th^ overall agreement 
figures for the three samples are 
remarkably similar, suggesting that 
whether informants are parents or 
attendants one can generally expect 
mentally retarded persons to agree 
with them slightly less than two- 
thirds of the time. Where questions 
of one type were asked of more than 
one sample, the similarity in agree- 
;«ent figures is impressive. For the 
most part, the conclusion^ drawn from 
analysis of agreement in the commu- 
nity children's sample apply equally 
to the data from additional samples. 
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Table 7.7: Agreement by Question Type: , All Samples 




1 of Questions 


% Agreement 


Question type 


Community Inst. Inst. 
Children Adults Children 


Community' Inst. Inst. 
Children Adults Children 


Yes-No (words) 


22 62 35 
10 10 i4 

/' 
f 

7 4 / 2 

r 

1 

1 

1 

4 / 
6 

5 • 26 1 
4 4 

1 

3 19 5 


65.7 60.6 63.0 

71.5 72.1 60.1 . 

65.6 71.4 "72.3 


Yes-No 

(with pictures) 


Either-Or 
(words only) 


jSither-Or 
, (pictures) 


58.1 

77.9 ; 

24.0 33.2 25.6 
64.3 81. c 

7.6 

77.9 79.9 ' 72.0 

•• 


Multiple choice 
(discrete) 


Multiple choice 
(quantitative) 


Open-Ended (factual) 


Open-Ended (numerical) 


Open-Ended (coded 
categories) 




61 126 . 56 


/ 

64.5 59.3 62.8 





ADDITIONAL INFORMATION FROM THE 
INSTITUTIONAL SAMPLES 

' ^es*-Jto_Questions in the Institution- 
alized Adult Sample 

In the' discussion of the commu- 
nity children's sample, it was 
suggested that a group of questions 
to which 50% of the responses of 
significant others were "yes" would 
give an improved^ estimate of re-, 
sponse validi^ty . \ The simple yes-no 
questions for theXinstitutionalized 
adult sample provide just such an 
estimate. For 22 simple yes-no 
questions in the community children 
sample, 68.8% of parents' responses 
were "yes," and agreement was 65.7%. 
For the 44 simple yes-no questions 
asked of institutionalized adults, 
47.7% of attendants' responses were 
"yes," and agreement dropped to 
57.7%. It is apparent that when 
agreement is not inflated by 
acquiescence, agreement for yes-no 
" questions is only slightly better 
than the 50% that would be expected 
if responding by both clients and 
their significant others were com- 
pletely random. 

Rules Questions 

Eight yes-no questions dealing 
with rules ("Are you allowed to stay 
up late at night?") were posed of 
institutionalized adults and yielded 
a pattern of responses quite dif- 
ferent from that typifying yes-no 
questions in general. For four 
questions, the wording was "Is it 
against the rules ... ?" and for the 
other fbnr, the wording was "Are 
you allowed...?" All questions 
dealt with behaviors that are 
generally prohibited. These ques- 
ti ^ns .were paired sq that the two 
sets of four dealt with precisely 
the same content but were worded 
oppositely. Agreement for the "Is 
it against the rules.^.?" questions 
was only 53.4%, with i35.3% of the 



comparisons disagreements in which 
the clients respc-ded "no" and the 
attendant "yes." Agreement for the 
"Are you allowed...?" questions 
averaged 79.9%, with 70.3% of com- 
parisons agreements in which both 
respondents answered "no." It 
appears that questions dealing with 
obvious rules and phrased "Are. you 
allowed...?" are unusually conducive 
to valid responding. 3y contrast, 
respondents appear to have mis\inder- 
stood the "IS it against the 
rules...?" questions and may have 
responded "no" thinking that tjhey 
were indicating that the beha\^ior 
was not allowed. Even more inter- 
estingly, agreement for these jques- 
tions was clearly not influenced by 
acquiescence. since most agreements 
consisted of "no" responses from 
both client and attendant. Retarded 
persons do not acquiesce to just any 
yes-no questions. (In fact, here 
the correlation between proportion 
of attendants responding "yes" and 
agreement was -.75, a direct 
reversal of the more usual trend.) 
These questions , in combination 
with the skills questions discussed 
earlier, suggest that the retarded 
are most likely to acquiesce when 
that is the socially desirable re- 
sponse but do not when "no" appears 
to be socially desirable. Clients 
appeared to have a clear sense that 
the activitic3s asked about were 
socially \inacceptable. 

Last-Option Position Bias in 
Either-Or Questions 

Within the either-or questix^ns 
asked of institutionalized adults, 
there was evidence for a weak 
position bias on the part of mentally 
retarded respondents, who showed a 
slight preference for the last- 
mentioned alternative. Asked to 
specify whether they are usually 
happy or sad, 91.2% of clients re- 
sponded "happy" when that was the 
last-mentioned alternative, as 
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opposed to 84.3% when "sad" was 
mentioned last. When asked if they 
are usually alone or with others, 
64.5% of respondents said "with 
others" when that alternative was 
mentioned last, as. opposed to 
51.5%* when " alone" was mentioned 
last. This response set tends to 
elevate agreement slightly when 
the last-mentioned alternative is 
the response preferred by attendants. 
However, this effect is very small 
by comparison with the effects of 
acquiescence. Agreement for either- 
or questions in the adult institu- 
tional sample supported our earlier 
conclusion that the validity ^of 
responses to either-or questions 
exceed^ the validity of -responses 
to yes-no ^questions. 

Open-ended Questions with Preset 
Categories 

Four questions ^sked of insti- 
tutionalized adults and their 
attendants were open-ended questions 
for which responses were coded into 
one of two preset categories. For 
example, all responses to the 
question "Who decides how you spend 
your money?" were coded into' the 
categories "somebody else" or 
"client." Average agreement for 
these questions was 81.6%, ranging 
from 70.6>^o 92.6%. (A fitth ' ' 
open-ended que^^on-^'How many 
people sleep in your bedroom?" re- 
quired numferical responses and re- 
sulted in agreement of only 7.6%. 
Interviewees were totally unable, to 
provide valid answers to this ques- 
tion.) The average agreement. figure 
of 81.6% for the four questions Ear 
exceeds that of 64.3% for the open- 
ended factual questions in the 
community children's sample. It 
appears that open-ended questions ■= 
soliciting simple information and 
using preset categories may be 
more promising than was concluded 
on the basis of interviews of 
community children. In fact, the 



unusually high agreement for these 
questions casts this format in a 
very favorable light, although it 
is still the case that many inter- 
viewees cannot answer such questions. 

Conclusions 

In short, the data from addi- 
tional samples modify the conclu- 
sions drawn from the community 
children saihple in a few ways. 
First, acquiescence does not occur 
to all questions. The "rules" 
questions showed that under specific 
circximstances, this response bias 
can be outweighed by other factors 
(possibly a social desirability re- 
sponse set). Second, either-or 
questions in some cases engender a 
last option position bias, which to 
some degree reduces the validity of 
responses to these questions. 
Third, open-ended questions with 
preset response categories may be 
somewhat more useful than was con- 
cluded on the basis of responses 
of community children. In general 
the conclusions drawn from the 
community children sample are 
supported by these further analyses. 

Correct-Incorrect Questions as a 
Measure of Response Validity 

ALL QUESTIONS 

Although client-significant 
other agreement served as our basic 
measure of response validity, a 
number of questions posed of mentally 
retarded respondents had known 
correct answers, allowing for an- 
other type of analysis of validity. 
Of nineteen such questions, the re- 
sponses to twelve were coded as 
either incorrect, partially correct, 
or correct, while responses to the 
remaining seven were coded either 
incorrect or correct. For the 
entire group of twenty, questions,. 
77.1% of clients' responses were 
correct or partially correct. For 
the twelve incorrect- partially-correct 
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questions, an average of 18.9% of 
responses were incorrect/ 19.9% 
were partially correct, and 61.2% 
were correct. For the seven ques- 
tions coded simply incorrect or 
correct, an average of 29.7% of 
responses were incorrect, and an 
average of 70.3% of responses were 
correct. Because such average 
figures are difficult to conceptu- 
alize, these cjuestions are dis- 
cussed on a more detailed level 
below. 



QUESTIONS ASKED OF ALL SAMPLES 

Two open-ended questions were 
asked in identical form of respon- 
dents in the three major samples 
included in our study: community, 
children, institutional adults, and 
^institutional children. The first 
question, . "What is your full name?" 
yielded the results depicted in 
Table 7.8. Answers were coded as 
partially correct if the respondent 
made an obvious attempt to say 
either the first or last name, and 
correct if both the first and last 
^names were stated correctly. In 
general, the data in Table 7.8 



reveal that about three-quarters 
of clients who can answer the 
question are able to give their full 
names correctly, and another quarter 
are able to state either their first 
or last name. Only one of the 108 
clients who responded appropriately 
to this question was unable to give 
at least a partially correct answer. 
In one sense, this seems somewhat 
encouraging. However, it is impor-' 
tant to remember that many clients 
were unable to respond to this 
question. In fact, when those 
who were asked but were unrelspon- 
sive to this question are considered, 
only 64.5% of all community chil- 
dren, 40.2% of institutionalized 
adults, and 50.0% of institution- 
alized children who were asked the 
question were able to correctly 
state both first and last names. 
Since one's name is in a sense the 
most basic piece of information 
that anyone possesses, consideration 
of this question could perhaps 
serve as a sobering foundation for 
our thinking regarding interview- 
ing retarded persons. If only 
around half of mentally retarded 
persons can correctly state their 



Table 7.8: Percentage of Correct Responses to "What is your full ncime?" 





Incorrect 


Partially 
Correct 


Correct 




Community children 


0.0% 


22.9% 


77.1% 




Institutionalized adults 


3.3% 


30.0% 


66.7% 




Institutionalized children 


0.0% 


26.7% 


73.7% 
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full names in response to a direct 
question, then the implicatic.i of 
the previous portion of this chap- 
ter, that the validity of the re- 
sponses of mentally retarded indi- 
viduals is frequently minimal and 
always in question, is no way sur- 
prising. The most optimistic in- 
terpretation of this information is 
that when retarded persons do 
respond appropriately to very simple 
questions, their responses can be 
expected to be at least partially 
correct. 

Additionally, the question "How do 
you spell your name?" was asked of 
the three samples. In response, 
74.5% of community children, 71.4% 
of institutionalized adults, and 
82.8% of institutionalized children 
have responses which were at least 
partially correct. This information 
perhaps pertains more to our re- 
spondents* language skills than to 
response validity per se. 

QUESTIONS ASKED OF TWO SAMPLES 

Two additional questions were 
included in the interviews with 
both community children and institu- 
tionalized adults. The first of 
these questions, ."Please write your 
name for me on this paper," was 
again more a test of language skills 
than of response validity per se. 



Among community children, 77.8% o£ 
respondents gave correct or partially 
correct answers, and 60.0% of insti- 
tutionalized adults were at least 
partially correct. / 

The second question was phrased f 
slightly differently for the two 
samples. "What is your address at / 
home?" was the wording for the comr 
munity children, as opposed to "Whkt 
is your address here?" asked of / 
institutionalized adults. Respphses 
of institutionalized clients were 
coded correct if they mentioned 
both the institution and the/city, and 
partially correct if they^mentioned 
one but not the other. Responses 
of community children were coded 
correct if they gave a mailing or 
street address, and partially correct 
if they gave a street name without a 
house number. The fraction of 
correct responses, depicted in Table 
7.9, gives us another solid insight 
regarding the issue of response 
validity. This question involves 
information which should be readily 
available io most persons, but it 
was apparently considerably more 
difficult than the "name" questions. 

ADDITIONAL QUESTIONS 

A number of additional factual 
questions were asked of only one 
sample. Three open-ended questions. 



Table 7.9: Percentage of Respondents Correctly Stating Their Addresses 





Partially 






Incorrect 


Correct 


Corrept 




21.4% 


14.3% 


64.3% 




25.0% 


16.7% 


58.3% 





Community children 



Institutionalized 
adults 
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coded incorrect or correct, were 
asked of institutionalized adults. 
These three questions , "What month 
is your birthday?" "What day is 
your birthday?" and "What year 
were you born?" resulted in 80.6%, 
77.8%, and 64.7% correct responses, 
respectively. Again, those clientis 
who were able to respond appropri- 
ately generally gave accurate infor- 
mation. However, if we take intq' 
account unresponsive persons, only 
28.2% of those clients asked the 
question were able to state their 
day of birth, 48.6% correctly named 
the month in which they weye born, 
and only 20.1% correctly n^amed their 
birth year. In short, mentally re- 
tarded respondents are often unable 
to correctly answer even very simple 
questions, but this inability usually 
takes the form of a failure to respond 
appropriately, so that those who -do 
respond appropriately are likely to 
also respond correctly. 

Four yes-no questions having known 
correct answers, all involving the- 
value of money, were included in the 
interview of institutionalized adults. 
These questions were included to 
assess subjects* understandings of 
money concepts. For the questions 
"Could you buy some candy with $100?" 
"Could you buy- a new bicycle with 
$1.00?" "Could you buy a new TV with 
$10.00?" and "Could you buy a record 
albvim with $10.00?" the fraction of 
correct responses was 38.9%, 60.5%, 
27.0%, and 92.3%, respectively. It 
is especially interesting to note that 
the questions for which the correct 
answer was "yes" had by far the 
highest fraction of correct responses, 
another instance of the effect of 
acquiescence. This factor makes it 
difficult to evaluate the validity of 
responses to these questions. The- 
fact that 50% of respondents would be 
expected to answer these questions 
correctly by chance and only 27.0% 
correctly answered the TV question 
casts serious doubt on whether re- 



tarded persons can be Expected to. 
answer questions about the value pf 
money. / 

Finally, two open-/ended questions 
were asked only of the institutional 
children. In response to the ques- 
tion »'What is /your birthday?" 66; 7% 
of children/gave correct responses, 
27.3% gave'' partially correct responses 
and 6.1% gave incorrect responses. 
For the question "What kind of place 
is this?" 61.9% responded correctly 
(by referring to the institution) and 
38.1% incorrectly. Once again j our 
clients showed a clear but limited 
ability to respond correctly to simple 
qr,estions, if they can respond' at all. 

CONCLUSIONS : 

As with the agreement data, the 
results of these factual questions 
must leave us pessimistic but not 
without hope. Mentally retarded 
individuals do tend to respond cor- 
rectly to these simple, straight- 
^forward factual questions. However, 
a discouragingly high proportion of 
respondents respond incorrectly when 
very fiandciinental information is re- 
quested of them: name, birth date, 
address^, and so on. Furthermore, 
when responsiveness data are con- 
sidered in tandem with information 
about validity, it is clear that only 
a small proportion of retarded persons 
can correctly answer ^even these "easy" 
questions. Examples of some of the 
difficulties we encoiantered are in- 
cluded in Sample 7.1. These correct- 
incorrect questions pro/ide a succinct 
summary of all our data regarding 
validity, in that they demonstrate 
that mentally retarded persons can, 
under some circumstances, provide 
valid information about themselves, 
but that the validity of the infor- 
mation -they can provide can never be 
taken for granted. 
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'. Sample 7.1 

A Sampler of Problems in Obtaining Basic Factual Information 



1. Q: What is your full name? 

A: (Lifted her foot and pointed to the bottom of shoe, where her name 
was written) 

2. Q: What is your birthday? 

A: I don't know. They got it in the record?. I'm 15. I thixik 
it ' s February . 

3. Q: What is your name? 

A: My name is Ronnie Peters. 

Q: How do you spell yojur name? 

A: RONKLTUWXYZ 

4. Q: What year were you born? 
A: Little Rock. 

5. Q: Do you get SSI? 
A: What? 

Q: Do you get SSI? 

A: SSI? Spell it! ! \ ' 

6. Q: What is your address\^here? 
A: 35944234 ^ \ 

7. Q: What is your full name'?^ ^ 
A: Huh? I don't got no name.. 
Q: Do you get SSI?.- 

A: I gots my bus ticket. 
Q : (repeated) 

A: I guess. 

Q: Do you have a family? 

A: No except my mama and daddy. That's all I have. 
Q: (repeated) 

A : No . 
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Agreement With Parents and 
Attendants as an Individual 
Characteristic 

To provide another perspective 
on client-rsignif icant other agreement 
we focused on 13 yes-no questions that 
were asked in precisely the same form 
to. institution children, community 
children, and institution adults. 
Nine of the questions were about par- 



ticipation in various chores and were 
accompanied by pictures, while the 
remaining four were simple verbal 
yes-no questions. An agreement score 
was calculated for each subject who 
had responses paired with significant 
other responses on at least half of 
the set of 13 questions. The agree- 
ment score could range from 0 ic 100%. 
We sought to determine whether agree- 
ment differed from sample to sample 
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when the' same questions were at issue 
and to explore individual differences 
in the tendency to agree with parents 
or attendants. 

Figure 7,2 presents the mean agreement 
scores for severely , moderately , and 
mildly retarded subjects in each of 
the three samples. As the figure in- 
dicates, there was some tendency for 
agreement with significant other to 
increase as IQ increased. A 3 X 3 
analysis of variance indicated that 
there was a significant difference 
between IQ groups (F (2, 129) == 3.62, 
p -03), but 'that, the three samples 
were not significantly different 
from each other and that IQ group 
and sample did not interact. Over- 
all, severely retarded persons 
agreed with their significant others 
70,5% of the time, moderately re- 
tarded persons 74,4% of the time, 
and mildly retarded persons 79.7% of 
the time on these questions. Follow- 
up t-tests with the three samples 
collapsed ind-^cated that severely re- 
tarded persons were significantly 
less likely to agree with their 
significant others than were mildly 
retarded persons r t^ (135) = 2.56,£ = 
.012. Although the interaction be- 
tween IQ group and sample was not 
significant, there were some variar 
tions in findings from sample to 
sample. Among institutionalized 
children, the severely retarded 
group was definitely deficient in 
comparison to the mildly retarded 
group, t (42) =2.66, p = .02. In 
the community children's sample, the 
difference between severely retarded 
and mildly retarded respondents 
approached significance (£ = ,056).. 
Interestingly , i there were no signifi- 
cant IQ group differences in the 
adult institution sample, and in 
fact agreement scores actually de- 
creased slightly as IQ increased. 
Altogether, then, this analysis 
revealed a tendency for agreement 
with significant other to be a func- 



tion of level of retardation, 
although it was certainly not a 
strong pattern even though it was 
statistically significant. Moreover, 
level of agreement did not differ 
greatly from sample to sample,-— despite 
the fact that the parents of children 
living in the community wight^ be ex- 
pected to be more knowledgeable in- 
formants than institution attendants 
responsible for many residents. Of 
course, it must be borne in mind that 
this analysis could only be conducted 
on yes-no questions, which, as dis- 
cussed previously, often appear to 
yield biased answers which nVay inflate 
estimates of validity as indicated by 
agreement with significant others. 
Nonetheless, for such questions one 
can apparently expect agreement with 
significant others, whether tliey are 
parents or institution staff, to 
average about 75% and to be weakly 
but positively related to the 
respondent':^ IQv^ 

To show more concretely the levels of 
agreement which were obtained. Tables 

7.11, 7.12, and 7.13 report for 
each sample the percentage of 
clients and significant others 
saying "yes" to each question and 
the percentage of cases in which 
answers agreed. As noted in Table 

7.12, there was only one instance 
in which the kappa statistic indi- 
cated agreement beyond the level 
expecteid by chance (cleaning floors 
in the adult institution sample) . 
At the same time, the percentages 
agreeing are exceptionally high on 
many other questions, and it must 
be borne in mind that extremely 
lopsided distributions in which 
virtually everyone says "yes" miti- 

^ gate against obtaining statistically 
significant agreement. The tables 
add still another perspective, for 
in some cases, despite relatively 
low agreement figures, the proportions 
of clients and significant others 
saying "yes" are very similar. For 
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Table 7.11: Agreement in Children's Institution Sample ; 



Activity 


% Yes according 
to clients 


/■ ' ■ 
%' Yes according 

to SO/' 


Percent 
Agreeing 


Kappa 


Watch TV ^ 


97.8 


97.8 


95.6 


-.022 . 


Listen radio, record player 


92.9 


100. 


92.9 . 


.0217 


Go to church 


89.1 


100. 


kl 


.000 


. Count money (knows how) 


88.5 


42.3 


\; 

38.4\ 


..•101, 


Set table 


75.6 


95.6 


■75.5 \ 


.081 

\ ' 


Do dishes 


90.7 


58.1 


- 62.8 


\ . 

\.145 


Clean the floor 


71;1 


95.6 


71.1 


.060 

' \ 


Dust furniture 


■ 83.7 


88.4 


76.7 


.034 ^ 


Make bed 


90.7 


90.7 , 


'86.0 


.170 


Pick up stuff 


93.2 


' 95.5 


88.6 


-.039 


Cook on the stove 


61.0 


7.3 


41.5 


.015 


Make sandwiches .(food) 


89.2 


24.3 


i 

29.7 ■ 


-.002 


Take. out trash 


97.7 


79.1 


81.4 


.164 


~ ' . J 
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Table 7.12:\ Agreement in Comunity Children's Sample 



Activity 



Watch TV 



Listen radio, record player 



Go to church 



Count nonay (knows how) 



Set table 



Do dishes 



Clean the floor 



Dust furniture 



Make bed 



Pick UP stuff 



Cook on the stove 



Make sandwiches (food)_ 



Take out trash 



% Yes according 
to clients 



98.? 




76.4 



76.4 



85.7 



89.1 



65.5 



89.1 



83.6 



% Yes according 
to SO 



100.0 



100.0 



85.5 



61.5 



76.4 



72.7 



74.5 



78.2 



82.1 



96.4 



52.7 



90.9 



89.1 



Percent 
Agreeing 



98.2 



90.9 



78.4 



53.9 



72.7 



69.1 



65.5 



65.5 



75.0 



.85.5 



47.2 



83.6 



80.0 



Kappa 



.000 
.000 
.212 
-.074 
.263 
.237 
.069 
.017 
.077 
-.057 
-.074 
.090 
.157 



ERIC 



MG 



Table 7.13! Agreement in Adult Institution Sample 



1 

Activity 


I Yes according 
to^clients 


■ 

% Yes according 
to SO 


Percent 
Agreeing 


Kappa 


watcn TV 




3/. J 




- 026 . 


Listen radio* record olaver 


89.7 


100.0 


89.7 


.000 


Go to church 


94.7 


100.0 


94,7 


, .000 


Pminf mnnou ftnnuc hr\w\ 
^UUIlU lUUiicy \MlUWb liUW/ 


71.9 


31.3 


53.1 


.194 


Dei tdUit 


79 5 


76.9 


61 6 


-.125 


r\f\ A^c)nCkQ 
UO QlSilco 


78 9 


' 78.9 


73.7 


.210 




78.9 


81.6 


81.6 


.420* . 


^••M^ £«4W*M^ l*l«VtA 

Dust iUrniture 


04* £ 


92 1 


81 5 • 


.128 


HaKe Dea 


2 


94 6 


83 8 •• 


-.078 


Pick UP stuir 


ujf 0 




78 4 


-.088 


Cook on the stove 


0/*0 


0*0 




- 003 


Make sandwiches (food) 


75.0 


69.4 


66.7 


.174 


Take out trash 


86.5 


97.3 


83.8 


-.047 













* Z = 1.77, significant at the ,05 level. 



example, while institutionalized 
adults and their attendants were in 
agreement only 61-1% of the time 
abput the client's involvement in 
setting the table, the 79.5% of the 
clients indicating involvement is 
not very different from the 76.9% of 
the attendants indicating involve- 
ment. In^BUch cases, the implications 
'of disagreement between clients and 
significant others are not severe if 
one is only interested in aggregate 
figures, for despite disagreement at 
the individual level, pictures of 
the group emerging from the two 
groups of informants are much alike. 
In other cases, however^ it is clear 
that one does not obtain the same 
• information from the two groups of 
informants. The most markedly dif- 
ferent pictures of the group were 
obtained on the questions about 
whether the client knows how to count 
money and cooks on the stove. Ac- 
cording to the clients themselves, 
for example, 88.5% of the institution 
children, 80.8% of the community 
children, and 71.9% of the institu- 
tion adults know how to count money. 
The corresponding figures based on 
significant other responses were 
only 42.3%, 61.5%, and 31.3%, respec- 
tively. Similarly, according to 
clients 61.0% of th3 institution 
children, 65.5% of the community 
children r and 67.6% of the institu- 
tion adults cook on the stove. The 
quite different figures provided by 
significant others were 7.3%, 53.7%, 
and 8.8%, respectively. In both 
cases, clients, by saying "yes" 
frequently, are apparently over- 
stating their involvement. These 
figures reinforce once more the 
variations in agreement associated . 
with individual content areas and 
the value of interpreting agreement 
from more than one perspective. 



CORRELATES 'of AGREEMENT 

Given the fact that acquiescence 
on yes-no questions has emerged as 



a problem in interviewing the men- , 
tally "retarded, we attempted to look 
at agreement with significant others 
in relationship to acquiescence. 
We correlated the agreement score 
described above with a dichotomous 
acquiescence score which simply in- 
dicated whether or not a client 
responded "yes" to both the question, 
"Are you usually happy?" and the ques- 
tion, "Are you usually sad?" To 
further explore the nature of agree- 
ment with significant other as an 
individual characteristic, we also 
correlated the agreement measure 
with IQ, sex, and a responsiveness 
score developed on >the basis of 22 
items used in all three samples. 
Table 7.14 presents these correla- 
tions for each of three samples. 

WhcV does this table reveal .about 
agy^fment? First, as was indicated 
in the previous discussion, hiaher 
IQ respondents were more likely to 
agree with their significant others 
except in the sample of institution- 
alized adults. ^Moreover, those 
clients who were able to answer a 
greater proportion of questions 
appropriately were generally more in 
agreement with their significant 
others, although again this relation- 
ship does not hold in the adult 
sample. (It should be noted that 
agreement scores were calculated 
only if a client had been able to 
respond to at least half the ques- 
tions in the set of 13, and the 
correlation coefficients are based 
only on those subjects who had 
scores.) In the children's institu-- 
tion sample only, itiales were more 
likely to agree with significant 
others than were, females, while in 
the. coramunity'^ children's sample this 
relationship was reversed,., in direc- 
tion, and in the adult "s^ple non- 
existent, leaving us uncertain about 
the relationship between sex and 
agreement. Finally, in the children 
institution sample, respondents who 
acquiesced were significantly less 
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Table 7.14: Correlations of Measures In Three Samples 







Sex 


Agree 


I 

1 

Response 


Acquiescence 






Inst, child 
Comm. child 
Instit. aauit: 


-.13 
-.02 


.40** 
.33** 
.02 


.63** 
.33** 
.31* 


-.40** 
-.30* 

-:3i* 




Sex 






-.28* 

.38** 
-.08 


-.02 
.11 
.01 


-.02 
-.06 
-.14 


Agreement 








.37** 

.20 

.07 


-.32* 
. -.13 
-.03 


Re spon s i vene s s 


1 c 








-.44** 
. -.21 
-.38** 



♦Significant at the -05 level 
♦♦Significant at the .01 level 

Note: In each, cell of the matrix, correlations 
institution children, community, and instituti 
is coded male = 1, female = 2. 

likely to agree with their signifi- 
cant others, but this negative rela- 
tionship was weak in the other 

^ samples. The relationship between ^ 
acquiescence and agreement is diffi- 
cult to unravel. Lower IQ clients, 
while they tend to disagree with 
informants, also tend to acquiesce 
a great deal, and as we have noted 
previously acquiescence may inflate 
agreement if informants often say 
"yes." Curiously, relationships 
were consistently stronger in pre- 

-it^ dieted directions in the children's 



in order moving downward are for the 
on adult samples, respectively- Sex 



institution sample than in other 
samples. Generally, trends in the 
same direction were evident among 
community children, but none of the 
relationships held among institu- 
tionalized adults. As this table 
also indicates^ IQ was quite consisr- 
tently related to responsiveness in 
a positive direction and to 
acquiescence in a negative direction 
(i.e., higher IQ subjects v^ere more 
responsive and less likely to acqui- 
esce) , and similarly, highly respon- 
sive subjects were less likely to 
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acquiesce than were less responsive 
subjects • 

Conclusions 

This chapter has examined the 
validity of answers given by mentally 
retarded interviewees in two ways; by 
comparing their answers with parallel 
responses given by attendants or 
parents , and by comparing their 
answers with known fact. As this 
analysis has indicated, establishing 
whether or not answers are valid, 
especially through the first approach, 
is problematic. We had no alternative 
most of the time but to look at each 
question individually and then try to 
detect patterns in the data for sets 
of questions. 

What can we conclude overall? We can 
conclude, first of all, that the in- 
formation provided by retarded pelrsons 
and the information provided by non- 
retarded informants about retarded 
persons is not necessarily the same. 
Indeed, in many cases one would obtain 
very different pictures of the needs, 
attitudes, and circ\amstances of re- 
tarded citizens from the two types of 
respondents. Our data suggest that 
their answers match approximately 
two-thirds of the time overall. The 
implications of this basic finding 
are serious. Mental retardation 
researchers (and practitioners as 
well, in their informal information- 
gathering efforts) have not been 
sensitive to this phenomenon. They' 
have often relied on one or other 
source of information, ass\aming^. but 
only assuming, that data from the 
two sources would be interchange- 
cible. To give an example, consider 
the literatui;^ on deinstitutionali- 
ion, which includes studies 
ed on client interviews and 
studies based on interviews with 
institution s^aff, houseparents, 
and other nonretarded informants. 
Studies based on different sources 
of data can be compared, but in view 
of the analyses presented here, one 
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must recognize that f:.ndings may 
differ from study to study because 
data were collected from different 
informants . 

At the outset, we warned that one 
can':<.t always assume that retarded 
per." >ns are wrong when they disagree 
with their parents or attendants. 
We did indeed find instances in 
which retarded persons and their 
significant others simply had 
legitimate differences of opinion. 
However, there were other instances 
in which we would point to response 
bias on the part of retarded persons 
as the cause of disagreement.. It is 
these latter cases that allow us to 
conclude that retarded persons are 
sometimes "wrong" in the sense that 
their answers do not reflect the 
state of things as much as they 
reflect the operation of response 
sets. The clearest case of all is 
the threat to validity of response 
to yes-no questions represented 
by acquiescence. As a general rule, 
retarded persons and their parents 
or attendants agreed on yes-no 
questions only to the extent that 
parents and attendants happened to 
say "yes" frequently, thus matching 
the fiequent "yes" responses of 
clients. This phenomenon raised 
serious questions about the very 
use of agreement figures as a measure 
of validity of response. In short, 
we had to take into account the 
fact that high agreement was not 
necessarily evidence of validity, 
not when we could identify other 
factors such as acquiescence that 
were artificially inflating agree- 
ment figures. 

We encountered similar problems in 
analyzing other types of questions. 
For examp].c3, since very few 
response^ were obtained to open- 
ended questions calling for en\amera- 
tions, hi^h agreement figures for 
categories of response to such ques- 
tions generally said little more than 
that clients and nonretarded infor- 

30 ' 



mants frequently did not mention 
something • Thus it must be recognized 
that many of the average agreement 
figures presented in this chapter are 
misleading indicators of^ validity of 
response. While we have "generally 
found that retarded people and their 
significant others agree about two- 
thirds of the time, that 
does not mean that responses are 
valid about two-thirds of the time. 

On the positive side, we did find 
either-or questions to have respect- 
ably high agreement figures without 
the pronounced response bias that 
characterized yefs-no questions. We 
also found high validity of response 
to multiple choice questions about 
basic factual information such as 
type cf residential setting where 
respondents chose among discrete 
response alternatives (compeCred with 
extremely low validity for multiple 
choice questions with quantitative 
response options) . And finally we 
found relatively high validity for 
factual open-endied questions that 
required a single answer rather chan 
requiring eniamerations of responses. 
At the same time, none of the agree- ^ 
ment figures was high enough that 
we can be completely confident in the 
information obtained from retarded 
persons. We must still conclude that 
their answers cannot be assiimed to \ 
be valid unless there is evidence 
to support that assumption. The 
inability of many retarded persons 
to provide basic factual information 
such as full name, birth date, and 
address simply reinforced that con- 
clusion. 



Is it possible to predict which 
retarded persons are likely to pro- 
vide valid answers? We attempted 
to answer this question by calcu- 
lating agreement scores for indi- 
viduals reflecting their tendency 
to agree with their significant 
others on a set of yes-no questions 
that were used in 'ill three major 
samples. We found that children 
living in the community were, sur- 
prisingly, no more likely to agree 
with the^r parents than institution- 
alized persons were to agree with 
their attendants, even though we 
had expected parents and children 
to be more likely to share a common 
set of facts and perspectives. 
While we foiind that higher IQ persons 
were more likely than lower IQ persons 
to agree with significant others, the 
relationship was not really strong 
enough that one could use it to pre- 
dict in advance whose answers will 
be valid. This matter needs to be 
explored further, since our findings, 
which were restricted to yes-no 
questions, might not generalize to 
other types of questions, particu- 
larly in view of the acquiescence 
problem for yes-no questions. Until 
further research is done, one must 
be wary of responses given by any 
retarded person, especially on the 
kinds of questions that we have found 
to elicit biased responses. 

In the next chapter, we will pursue 
tKe topics of response bias and 
response validity through systematic 
comparisons of alternative ques- 
tioning approaches. 
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Chapter 8 



WHAT INTERVIEWING APPROACHES 

ARE MOST USEFUL? 



A major focus of this report haS 
been evaluation oi different kinds 
of questions according to the cri- 
teria of responsiveness, reliability 
and validity as. indicated by agree- 
ment with parents and attendants. 
To facilitate this evaluation, all 
of tt>«^ interview schedules used in 
the various studies included spe- 
cific comparisons of alternative 
ways of soliciting~the samo-infor^--. 
mrxtion, head- to-head competitions 
among different ,formats and phras- 
ings applied to the same content 
area. These comparisons were done 
specifically to identify question- 
ing techniques that optimize re- 
sponsiveness , reliability, and 
validity. 

To illustrate the approach used, 
consider the comparison summarized 
in Sample 8.:.. In comparing two 
questions, in this case a yes-no 
question and a parallel open-ended 
question, we first apply a criter- 
ion of responsiveness. We would 
generally prefer a question that 



can be ai.r^wered by many persons to 
one that relatively few, can answer. 
In the example, the institutional- 
ized adults studied found it con- 
siderably easier to answer the yes- 
no question than to answer the 
open-ended question, a finding 
which is entirely consistent with 
our analyses of responsiveness as 
a function of question format (see 
..Chapter ,...6), • 

A second criterion centers on re- 
liability in the sense of consis- 
tency between answers to the two 
alternative questions. The con- 
sistency of responses to these two 
questions was relatively low. 
While that alone does not' give us a. 
basis for preferring one question . 
over the other ; it is also apparent^ - 
that there is a systematic trend 
in the inconsistency. Namely, clients 
are highly likely to say, "Yes, I 
decide," in response to the yes-no 
question while they are mentioning • 
someone other than themselves in 
response to the open-ended question. 
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Sample 8.1 

COMPARISON OF ALTERNATIVE FORMATS 
Yes-no versus Open-ended 
(Do you decide what chores you do? versus Who decides 
what choies you do? — Adult Institution Sample) 

1 . Which Question Yields Higher Responsiveness? 

Percent Responding Appropriately: Yes-no 54.4% 

Open-ended 34.5% 

2. To What Extent Are Answers Consistent? 

With answers coded client decides versus someone else, 
decides, percent consistent = 56.3%. All of the rest 
.(43.7%) are instances of "Yes, I decide" on yes-no and 
mention of someone else on open-ended — i.e., acquiescence, 

3. To What Extent Do Clients And Significant Other Agree? 

For yes-no , 34.6% aqiee. Remaining 65.4% are cases in 
which client says, "Yes, I decide," while attendant men- 
tions someone else in response to open-ended question. 
jt. 

For open-ended , 88.2% agree. 
CONCLUS ON: While more subjects can answer the yes-no question, 
acquiescence invalidates resp6nse. Open-ended ques- 
tion is preferred on grounds of validity. 
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This suggests that acquiescence in 
response to the yes-no question is 
operating and is inflating the 
number of respondents who claim 
they make their own decisions. 

Finally f we can apply a criterion of 
agreement with significant others 
(parents or attendants) in decid- 
ing which question is the best 
approach to use. Due to the acqui- 
escence detected in the consistency 
analysis f agreement with attendants 
is lower for the yes-no question 
than for the ^open-ended alternative . 
On the basis of the agreiement 
criterion, the open-ended format 
is therefore preferable. 

As is true in many of the analyses 
to follow, the application of 
these three criteria often involves 
trade-offs. While the yes-no ques- 
tion is to be preferred on grounds 
of responsiveness, it is clearly 
deficient on grounds of validity 
because of the strong influence 
of acquiescence. Since there is 
no point in getting answers if 
they are not trustworthy as valid 
answers, we must ultimately prefer 
the open-ended question in this 
case. Yet neither alternative is 
entirely attractive and use of the 
open-ended question will ultimately 
mean that data will be obtainable 
only ' f'rorti ' the ' itior ve^ segitient 
of the population under study. 

The comparisons which follow are 
often more complex than the ex- 
ample, but they follow the same 
general logic. While it will prove 
impossible on some occasions to 
clearly establish one\altemative 
as superior to another, we will • 
constantly be trying to determine 
w>->ich ways of asking questions 
^L^ir to work best in hopes of 
.uhing empirically bas^ ; 

xccoiBi'iendations for persons ^.^cc 
__estei X : int erviewing retarded 

individuals for research or program 

planning purposes. 



The chapter will consider in turn 
three major topics in question de- 
sign. First, we will report on 
several comparisons of questions 
whose wording differed but whose 
formats were the same, ^ asking the 
question: To what extent are the 
responses given influenced by the 
manner in which a question is 
worded? Secondly, we will briefly 
consider a procedural issue relevant 
to the use of open-ended questions: 
What are the relative merits of 
simply asking the question as op- 
posed to asking the question and 
then persistently probing with 
"What else?" in hopes of eliciting 
more information? This question 
is significant given our difficul- 
ties eliciting much information in 
response to open-ended questions. 

Finally, we address perhaps the 
major question underlying our 
studies; Which question formats 
appear to be associated with the 
greatest responsiveness, relia- 
bility and validity of response? 
We began our studies predicting 
that we would have difficulty get- 
ting answers to open-ended questions 
and that more structured questioning 
approaches might be more effective. 
In view of the limited verbal skills 
of many retarded persons, we also 
sought to deterrnine^^ w^^ 
use oF pictures in interviewing / 
might facilitate answering. ;we(will 
begin describing our comparisons of 
alternative formats by considering 
. the use of the multiple choice for- 
mat in gathering information about 
activities. Next, we will consider 
yes-no questions, comparing them 
with open-ended and either-or for- 
mats, as well as assessing the ef^ 
fects of pictures as a means of 
clarifying yes-no questions. Finally, 
we will analyze responses 
to either-or questions and explore 
th e potential value of using pic- 
tures as response alternatives in 
this format. 
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Most comparisons of alternative, 
formats were conducted a niamber 
of times with different content 
areas. Because of this repetition, 
it would be unwieldy to present all 
data comparing two formats. Thus, 
we will follow the general strategy 
of selecting one comparison con- 
sidered to be representative of 
the others, and presenting that 
comparison in some detail. Other 
comparisons of the same type will 
be cited briefly to supplement in- 
formation from the illustrative 
comparison. When xesults are con- 
tradictory or inconsistent, v • will 
attempt to represent that face ard 
explore reasons for it. ^ 

Question Wording 

Does the way in which a question 
is worded affect the ability of 
mentally retarded persons to answer 
and the kinds of answers they ^ l.ve? 
While we devoted more attention to 
the effects of question format, 
we did compare a few alternative 
wordings of questions posed ip the 
same format. (In the alternative 
formats section we will be discuss- 
ing several more instances in which, 
as part of a broader comparison, 
oppositely wojrded questions with 
the same format were evaluated) . 

USE OF EXAMPLES 

Consider first the strategy of 
including in a qiaestion examples of 
the types of responses one is solic- 
iting. The use of examples (e.g., 
baseball or football as examples of 
the class of sports) might be ex- 
pected to aid mentally retarded 
persons in understanding a question 
by making the question more concrete. 
Alternatively, such examples might 
bias respondents, leading them to 
name the items used as examples 
more than they ordinarily would. 



Illustrative Comparison 

Two questions asked institu- 
tionalized adults to report if 
they played any indoor games , and 
if so, which games. One question, 
"Do you play any games indoors, 
like cards or checkers? (if yes) 
Which ones?" gave examples of 
indoor games. The other, "Do you 
play any games indoors? (if yes) 
Which ones?" solicited identical 
information but did not use ex- 
amples. Analysis will be limited 
;:o the effect of examples on re- 
sponses to the open-ended portion 
of each question. 

Tlie use of examples increased 
responsiveness to the open-ended 
follow-up question. The percentage 
responding appropriately was 64.5% 
for the question with examples, as 
oppos-iJ to 52.0% for the question 
without examples. 

The x*^\' of examples also influenced 
the go.?.:B mentioned in response to 
the open-ended question. Specif ical 
ly, only 22.2% of nine adults 
responding to both open-ended ques- 
tions mentioned checkers after the 
plain yes-no questionr while 66.7% 
mentioned checkei;:s after the ques- 
tion in which checkers was included 
as an excimple, yielding a consis- 
tency fig\ire for this category of 
■'only '55'.S%; '"^Sim fespbhdehts''^ 
were more likely to mention cards 
in response to the question in- 
cluding cards as an example (55.6%) 
than to t.he plain question (22.2%), 
producing a consistency figure oi 
only 44.4%. Consistency— for_±he_ 
remaining categories was generally 
high and was achieved primarily by 
not mentioning a game both times. 
In short, respondents were far 
more likely to mention a category 
if that category was cited ^s an 
example. 

—Overall agreement_between_client 

cind attendant across the seven 
C '.^ cjgo) ies of response to the open- 
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ended question was 75.5% for the 
plain question and 72.4% for the 
question with examples. However, 
percentages of cases in which both 
mentioned a category averaged only 
10.2% and 4.8%, respectively, with 
most categories involving no such 
instances. For the critical 
categories used as examples, find- 
ings were inconsistent. For the 
checkers category, the use of ex- 
amples' apparently decreased agree- 
ment by inducing mentally retarded 
respondents to overclaim checkei-s; 
agreement was 85.7% for the plain 
question and 60.0% for the question 
with examples. In the latter case, 
40.0% of client-attendant pairings 
involved the client mentioning 
checkers while the attendant did 
not. This trend was not apparent 
for the cards c tegory; agreement 
averaged 42.9% after the plain yes- 
no question and 53.3% after the 
question with examples, and incon- 
sistency was mixed rather than 
biased in one direction. 

Additional Comparisons 

; Two other pairs of questions 
asked of institutionalized adults 
examined the effects of examples. 
One pair asked about the client's 
participation rports , using 
baseball and f''^^'->''-^an as 
in one ph^^ and another asKed 

about arts and crafts, mentioning 
ceramics and painting as examples. 
The effect of the use of examples 
on responsiveness to the open-ended 
follow-up is unclear. For the 
crafts question, examples apparently 
increased responsiveness from 41.7% 
to 59.3%. Conversely, the sports 
question without examples generated 
greater responsiveness (75.0%) 
than the parallel question with 
examples (57.7%). 

These additional comparisons did 
sx:^port__the.-j£inding - t ha t c iting. 



tioned by mentally retarded persons. 
For the four examples used in these 
two pairs of questions, 56.1% of 
respondents, on the average, men- 
tioned the activity when it was 
included as an example, versus 44.3% 
who mentioned the activity when no 
examples were given. The pattern 
of inconsistencies generally re- 
flected this bias toward overre- 
porting an activity used as example. 

Of these two pairs of questions, only 
the sports questions were asked of 
attendants. For the football cat- 
egory, agreement was 55.6% for 
the question with examples and 57.1% 
for the plain question. In both 
cases, most disagreements occurred 
because the client mentioned foot- 
ball and the attendw^t did not- 
Results for the baseball category 
were far more striking. Agreement 
was only 11.1% for the question 
with examples, and 77.8% of com- 
parisons were inconsistencies in 
which the client mentioned baseball 
and the attendant did not. For 
the plain question, agreement was 
far higher (71.4%). Thus, as for 
the indoor games questions , the 
use of examples decreased agree- 
ment for one of the activities 
used as an example but not for 
the other . 



an example increases the frequency 
with whi:.?^ that activity is men- 



Conclusions 

The use of examples in the intro- 
ductory question had important ef- 
fects on clients' responses. Al- 
though the effect on responsiveness 
to the open-ended follow-up ques- 
tion is unclear, examples did in- 
crease rhe likelihood that respon- 
dents would mention the example 
activities. The existence of this 
biasing effect is clear from the 
consistency data, and client-attendant 
agreement f igiires suggest 
that the use of examples sometimes 
- -induGas-over c 1 a i m i n gJay-jnen t a 1 ly_ 



retarded persons, reducing response 
validity '^^ appoars that the use 
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of examples with open-ended ques- 
tions only added ' to the problenis 
inherent in the basic questioning 
technique. Examples appeared to 
arm interviewees with ready an- 
swers that they otherwise would 
not have had. 

USE OF ALTERNATIVE TERMS 

We also examined the issue of 
question wording by testing the 
extent to which the language used 
to phrase a question affects re- 
sponses. This strategy was ap- 
plied with four pairs of ques- 
tions about rules; one member of 
each pair asked "Are you allowed 
to...?" naming a specific be- 
havior f while the other mr^mber 
asked "Is it against the rules 
to...?" naming the same behavior. 

Illustrative Comparison . 

Two such questions, asked of 
institutionalized adults, were 
i"Are you allowed to hit people?" 
and "Is it against the rules here 
to hit people?" There was no evi- 
dence that this difference in 
question wording altered respon- 
siveness, which was 70.7% for the 
"against" wording and 70.2% for 
the "allowed" wording. 

Overall consistency for these ques- 
tions was 4376%,"^ a^^ 
than that expected if responses 
to both questions were random. 
The great majority of respondents 
(87.2%) arswered the "allowed" ques- 
tion "nor" while responses were 
almost evenly split between "yes" 
and "no" in the case of the 
"against" wording. A very large 
fraction of clients, 46.2%, thus 
contradicted themselves by re- 
sponding "no" to both questions. 
This is in startling contrast to 
the more common result, where 
acquiescence on oppositely ^^rded^ ^ 
questions causes ~a' la:rg^^ropoYti^^^ 
of "yes-yes" self contradictions. 



cind it suggests that acquiescence 
cannot always be expected on yes- 
no questions. The most jj^robable 
explanation is that in the case 
of the. "allowed" questions, "no" 
was not only the correct response 
(since hitting people is clearly 
prohibitedi. , but was also si\ highly 
ingirained socially desirable re- 
spv-)nse. This response pattejp was 
carried over to the "against"^ ques- 
tions, probably because clients mis- 
understood the question and be-r 
lieved that "no" indicated that\ 
hitting people was forbidden. This 
interpretation, admittedly spec-\ 
ulative, accounts for the differences 
in the marginals and the preponder- 
ance of "no-no" self contradictions. 

For the "against" wording, agree- 1 
ment with significant others was 
51.3%. Nearly- all agreement re- 
sulted because the client said 
"yes" (hitting people is against the 
rules) and the attendant concurred. 
Almost all disagreements (43.6% of 
comparisons) involved a "no" response 
by the client when the attendant 
indicated that hitting was forbidden. 
This again supports the idea that 
many clients misunderstood the 
question and attempted, through "no" 
/ responses, to give a socially 
desirable response. 

Eo_r-,,the...."allQW e.dJ^J^^ 

wa"sT6.3%', with all agreCTient be- 
curring because client and attendant 
both said "no" (not allowed). 
Again, the major conclusion is that 
these questions, in direct contrast 
with the remainder of the yes-no 
questions included in our inter- 
views, did not give rise to acqui- 
escence, perhaps because clients 
had internalized strong negative 
evaluations of hitting. 

Additional Comparisons 

Itaee additional pairs of ques- 

— -tit5ns-posed-of-inst4^tionali2:ed_ 

adults pitted "Is it against ...?" 
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questions against "Are you ai- 
lowed,,.?" questions. The activ- 
ities involved in these questions 
were Calling people ugly names, 
leaving without asking, and stay- 
ing up late at night. Response 
patterns to the first two ques- 
tions were virtually identical to 
those for the "hit people" ques- 
tions, but responses to the last 
pair of questions differed great:|.y 
from responses to the other three. 

Mean responsiveness for the three, 
additional "against" questions was 
68,4%, compared to 66,1% for the 
three "allowed" questions. This 
supports the conclusion that oppo- 
site wording did not alter respon- 
siveness. Consistency figures for 
the "ugly names" and "leave without 
asking" questions were 40,0% and 
57,2%, respectively, with incon- 
sistency predominantly due to re- 
sponding "no" to both forms of the 
question. Consistency for the 
"stay up late" question was 38,2%, 
with 23,5% of comparison inconsis- 
tencies due to saying "yes" both 
times, and 38,2% inconsistencies 
due to sayi.ig "no" both times. 
Clearly, staying up late is less 
strongly prohibited than the other 
behaviors, leading to far more 
responses by clients indicating that 
staying up late is allowed. It is 

-in teres ting-that^this_f_actor^d^^ 

consistency. 

For the three additional "against" 
questiions, average agreement with 
attendants was 54.1%. Agreement 
was far higher, averaging 80.8%, 
for the "allowed" wording. In 
the latter case, clients' "no" re- 
sponses coincided with the preferred 
response of attendants. For the 
"against" questions, clients' at- 
tempts to indicate prohibition 
through "x.c" responses operated in 
opposition to attendants' responses, 
reducing agreement. This is sup- 
pdr ted" by ~ the" ob s erva tion~that^ 
disagreements for the "against" 



question (an average of 32.5% of 
client-attendant pairings) involved 
a "no" response from the client and 
a "yes" response by the attendant. 

Conclusions ^ 

Both consistency and agreement 
for these questions can be easily 
interpreted as arising from a 
strong tendency to say "no" on the , 
part of clients. Why does the usual 
yea- saying not appear? The simplest 
explanation is that there is a strong 
motive to give socially desirable 
answers; that is, it is highly de- 
sirable to express the xiriderstanding 
that prohibited behaviors are pro- 
hibited. This accounts for the high 
proportion of "no" responses to 
the "allowed" questions. Respondents 
appeared to understand the rules well 
enough to give a negative response 
to negative behavior. Whether they 
understood the "allowed" phrasing 
is less clear. One must assume that 
many clients misunderstood the 
"against" questions, believing that 
a "no" response was again the way to 
express disapproval /of \andesirable 
behavior. This interpretation at 
least raises the possibility that 
much of the acqu/fescence observed 
throughout our interviews may be 
partially accounted for by social 
desirability, for questions were 
-:^:^typi<;ally^worde(Lso_j^^^ 

the socially desirable response. 
It may be that a tendency toward 
agreement with socially accepted 
views (see discussion of Rosen et 
al,, 1977) is in fact t^e general 
response set in mentally retarded 
persons, and that acquiescence 
to most yes-nb questions and nay- 
saying to the rules questions are 
simply specific manifestations of 
that broader tendency. Alterna- 
tively, respondents may have been 
so well-verged on the prohibitions 
asked about in these questions 

th?4- ^y^f^^r- n.c;ual ten dencies toward 

acquiescence were overcome. 
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Regardless/ of the factors underly- 
ing clients' responses r one major 
conclusion emerges clearly from 
these comparisons: the picture of 
a group* of mentally retarded persons 
which is derived from interviews 
differs as a function of questidn 
wording. For the four "against" 
questions, an average of 61.2% of 
clients indicated that the behaviors 
are prohibited. For the four 
"allowed" questions, an average of . 
75.4% of clients indicated that the 
behaviors are prohibited.. In 
neither case do clients agree fully 
with attendants, about 85% of whom 
indicated that the stated behaviors 
are prohibited. In short, because 
the responses of mentally retarded 
individuals to yes-no questions 
are not entirely valid, the picture 
of the saruple derived from inter- 
view data depends largely on how 
questions happen to be worded. 

SLIGHT CHANGES IN QUESTION WORDIft^G 

Serendipity gave us some insight 
into the effects of question wording 
thatcvwe would not otherwise ha'^^e had. 
A typographical error caused us to 
ask institutionalized adults /two 
closely related questions, "mxo 
decides what chores you do?7 and 
"Who decides what chores td do?" 
The apparent effect of this change 
in questiqn^_wordi,^g jp^^ 
was startling. 

Responsiveness to these /two open- 
' ended questions was identical at 
34.5%. Regarding consistency, the 
fact that the first 
questions was repeateflN?# the two 
forms of the interview gi^es us a 
baseline estimate oy response 
reliability over a period of^ a 
week was 83.3%. Fot the two 
comparisons between the different 
wordings of the Question, consis- 
tency figures w^re much lower - 
62.6% and_ 73.356. While the 

— number^-of-reMons— respending-wa-s 

small, the comparison of these fig-- 



utes with the 83.3% figure for test- 
retest relicibility suggests that 
very ^mall changes in question word- • 
ing'may have effects on clients' re- 
sponses. 

Agreement figures for the two "you 
do" /questions were 88.2% and 75.0%. 
Agreement for the "to do" wording 
was' considerably lower (70.6%) . 
Possibly the "to do" question was 
more ambiguous. 



conclusions 

Clearly, even jvery slight changes 
/in question wording can have impor- 
tant effects on the reliability and 
validity of the reisponses of mentally 

retarded respondents. 

i 

i 

USE OF QUANTITATIVE QUALIFERS 

Included in follow-up interviews 
with the severely retarded sample 
were five yes-no questions and one 
either-or question with picture al-- 
tematives, all dealing with whether 
or not the client makes his or:, her bed. 

The questions, which used quantita- 
tive modifiers to introduce slight 
differences in meaning, were as 
follows: 

Do you always make. your bed here 
in the cottage? 

Most days, do you make your bed 
here in the cottage? , 
-Do_y_oja.ijasuaJLlv^ 
here in /the ccjttage? 
Do you 'sometimes make your bed 
here in the cottage? 
Do you ever make your bed here 
in the cottage? 
This picture shows a boy/girl 
who makes his/her bed here in 
the cottage. This picture shows 
a boy/girl who does not ever 
make his/her bed here. Which 
boy/girl is most like you? 
point to the picture. 

Responsiveness for the five yes-no 
questions ranged from 79.3% to 



Table 8.1 % "Yes" Responses to Bed-*making Questions 



— ^ 1 

1 

Question ' 


% Yes 


N 




Always 


\ 
I 


96.0 


25 




Most days 
usually 






100.0 
96.2 


26 
26 




Sometimes 

Ever (Yes- no) 

Ever (Picture either-or) 


92.0 
100.0 
77.8 


25 
25 
27 





89.7%, averaging 84.4%. There is 
no indication that responsiveness 
differed systematically as a fxanc- 
tion of the quantitative modifier 
used. Responsiveness for the 
either-or question using pictures 
was 89.7%. 

Regarding consistency, the best 
strategy for extracting meaningful 
information from tliiese comparisons 
is a kind of hierarchical analysis; 
if a client responds "yes" to the 
"always" question^ then he or she 
must necessarily tespond "yes" to 
the other questions to remain 
consistent. Table 8.1 shows the 
percentage of "yes" responses to 
the six' bed-making questions. As 
can be seen, no^ more respo'^^-c^ncs 
answered "yes" to the "usually" 
or "sometimes" question than to the 
"always" question. ,This calls 
into question the validity of re- 
sponses. Although it is logically 
possible that clients virtually 
all make their beds every day, re- 
sponses to/ the pictorial either-or 
question suggest' otherwise. Be- 
cause rest>onses/ to these questions 
represented subtle quantitative 
differences, agreement with sig- 
nificant othets is not a useful 
index of validity. 



Conclusions 

Perhaps the conclusion best 
justified by the data is that clients 
were relatively insensitive to the 
quantitative shadings involved in 
these questions - they tended to re- 
spond in the affirmative reguardless 
of the quantitative modifier used. 
This suggests that laborious efforts 
to refine questions through quanti- 
tative modifiers may not have the 
effects that question designers in- 
tended them to have. 

SPECIFIC VERSUS VAGUE WORDING 

Two comparisons of open-ended 
follow-up questions in the institution- 
alized adult sample compared general 
or vague question "wbrding with a more - - 
specific wording which restated the 
main. content of the question. These 
comparisons suggested that type of word- 
ing had little effect on consistency or 
agreement, but had a clear effect on re- 
sponsiveness; the specific wording max- - 
imized responsiveness. For example, as 
a follow-up to a question about attitudes 
toward. living in the institution, clients/ 
were asked either "Why do you feel- 
that way?" or Why do you feel that way 
about living here?" responsiveness to 
the follow-up using the, former word-- 
ing was 26.2%, as compared to 54.1% £or 
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the latter (more Specific) wording. 
Results were identical for a follow-up 
question dealing with clients* 
attitudes towards the food, with the 
responsiveness figure favoring the 
more specifically worded question, 43.9% 
to 30. 6%, In short, it is clear 
that/ if maximizing responsiveness 
to -open-ended follow-ups is the 
^des'ijred goal, the interviewer should 
Mse [wordings which repeat the main 
content of the question. This 
strsltegy does not appear to lessen 
validity in any way, and it has the 
decided advantage of reminding 
respondents of the question content. 

THE USE OF PROBES WITH OPEN-ENDED 
•QUEi^TlONS 

^As the data presented thus far 
suggest, the researcher or program 
^planner' who wishes to interview 
mentally retarded individuals faces 
a dilemma: highly structured ques- 
tion formats such as yes-no ques- 
tions maximize responsiveness but 
tend to introduce response biases 
which seriously reduce response 
validity; less structured formats 
such as open-ended questions, on 
the other hand, can be answered 
by few> and elicit little informa- 
tion from them. In order to find 
ways to improve open-ended questions, 
we at times probed "what else" as 
respondents answered ^.hem, hoping 
"to- increase-the-amount-of— informa- 



tion gained. The following section 
is a comparison of responses to 
open-ended questions using such 
probes and to otherwise identical 
questions with no probing. Because 
the probe follows the first re- 
sponse, the formats are identical 
with respect to the percentage of 
clients responding appropriately 
to the question, and analysis of 
responsiveness is omitted from the 
following discussion. 



Illustrative Comparison 

Institutionalized adults were 
asked the questions "What do you 
usually do for fun when you are by 
yourself?" and "What do you usually 
do for fun when you are with someone?" 
(probe: "What else?" until client 
can think of no more) . 

Consistency of responding to these/ 
questions was evaluated in terms of 
both the number of responses given 
to each question and the content of 
the particular responses given. In 
terms of numbers of responses, clients 
averaged 5.0 responses to the ques- 
tions with probe and 2.9 responses 
to the question without probe. 
Clearly, the use of the probe in- 
creases the number of responses 
given by the clients who could 
answer the questions. 

Consistency of response content, 
averaged over the ten categories 
into which responses were coded, 
was 80.0%. However, most of this 
observed consistency occurred 
because -clients did not mention a 
category in response to either for- 
mat; only 8.6% of comparisons were 
consistent in that the same category 
was mentioned both times. Of more 
interest is the fact- that most 
inconsistencies occurred when the 
client mentioned a category in 
response to the question with 
pr obe and failed to mention the 



same category when no probe was 
given. Of all comparisons, 15.0% 
(or three-fourths of all incon- 
sistencies) were inconsistencies 
of this type. 

Client-attendant agreement can also 
be evaluated either by the number 
of responses given, or by the re- 
sponse categories used. For. the 
question without probe, clients 
averaged 2.5 responses versus 2.7 
for 'attendants. For the question 
with probe, clients averaged 4.7 
responses versus 2.6 for attendants. 
Thus, the use of the probe induced 
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clients to mention far more acti- 
vities than did their attendants; 
this finding at least suggests 
that the probe technique may, by 
placing demands on the client, cause 
overreporting . 

Agreement between attendants' and 
clients' response to the question 
without probe, averaged across the 
ten response categories, was 82.3%. 
For the question with probe, 
average agreement across the ten 
response categories was 74.6%. Thus, 
the use of the probe apparently re- 
duced agreement, most disagreement 
occurring because the client men- 
tioned a category which was not 
mentioned by the attendant. All 
client-attendant disagreements in 
the probe format (17.7% of all paired 
responses) were inconsistencies of 
this'^type. In short, the use of 
the probe reduced agreement by in- 
ducing clients to mention categories 
not mentioned by attendants. 

Additional Comparisons 

Institutionalized adults were 
asked three additional pairs of open- 
ended questions comparing the prob- 
ing approach with identical ques- , 
tions without probing. These ques- 
tions asked, "What do you and your 
friends usually do together?" 
"Where do you usually get your 
money?" and "What are they teaching 
you there y"^' a question which was 
asked only of respondents who had 
answered "Yes" when asked "Do you 
go to school or take classes?" 
Results from additional comparisons 
of the probe and no probe formats 
were remarkably similar. In 'each 
and every case, mentally retarded 
individuals gave more responses 
when the interviewer probed "What 
else?" after each response and' in 
each case the use of the probe de- 
creased agreement. 
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Conclusions 

A remarkably clear pattern 
emerged from these comparisons. 
First, the use of the probe tech- 
nique was effective in eliciting 
more responses from mentally re- 
tarded respondents. Unfortunately, 
this had the effect of reducing 
agreement. It appears that the 
probe created a demand which was 
met by clients, regardless of 
whether the information provided 
was valid or not'. The net effect 
is that probing seems to cause 
inappropriate oyerresponding, re- 
ducing response validity. There 
was also some evidence that under- 
reporting occurs when no probe is 
used. Possibly, the optimal ap- 
proach would be to probe following 
one or two responses, rather than 
continuing to probe indefinitely . 

The disastrous effects of probing 
ad infinitum were clear in several 
of the interviews. One [severely 
retarded male , asked about^ what he 
and his friends usually do to- 
gether, apparently felt compelled 
to repeat himself in response to 
each, "What else?" His responses 
were "play round," "play aroxind," 
"go outside and play," and "go 
play," in that order. Clearly 
no new information was obtaijied; 
the only effect of probing was to 
force him to change his wording 
a_little in hopes of convincing 
the interviewer that he had new 
news, /bother, in response to 
the question, "Where do you get 
your money?" ' said, "I work." 
However, vhe then interpreted the , 
"where else?" probes that followed 
• as asking Tvhere else he worked, 
and then responded, in turn, "at 
the kitchen," "cottages," and "in 
here . " Still others , interpreting"^ 
the probes as a license to play 
word games, ran off lists of over 
10 activities, clearly doing so^ 
to meet the interviewer's demands . 
v'ith each naw probe. One, though 
iie repeated himself spmewhat, 
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came up with the staggering total 
of 27 responses to 27 "What else ' s. " 
His last response had to do with 
cleaning, and from then on he took 
the "What else's" to refer to other 
things he cleaned, giving the inter- 
viewer a much-welcomed opportunity 
to go on to the next question. 

Multiple-Choice Questions 

THE MEANING OF QUANTITATIVE RESPONSE 
OPTIONS 

We used many multiple choice 
questions offering response alter- 
natives like "a lot" and "some," 
which differ along a quantitative 
dimension, in order to collect in- 
formation about extents of involve- 
ment in various activities. The 
many problems we encountered in the 
use of quantitative multiple choice 
questions in interviewing mentally 
retarded persons may be a result 
of the difficulty those individuals 
have in attributing consistent mean- 
ing to quantitative words and 
phrases. Ones comparison will serve 
to illustrate this phenomenon. 
We asked institutionalized adults 
"HOW many friends do you have? A 
lot, some, not many, none?" and 
simply "HOW many friends do you 
have?" Respondents were more re- 
sponsive to the multiple choice 
question (55.2%) than to the open- 
ended question (39.7%). 

Regarding consistency, the numbers 
offered in response to the open- 
ended item did not correspond well 
to the categories endorsed in 
responding to the multiple choice 
question. For example, the 64.7% 
of respondents who answered- from one 
to six friends on the open-ended 
question answered anywhere from 
"none" to "a lot" when asked the 
multiple choice form. Among respon- 
dents who said they had seven or 
more friends in response to the 
open-ended question, responses to 



the multiple choice form wtere divided 
between "a lot" and "not ma^ny." 

in view of the fact that thkse quan- 
titative terms appeared to have yeiry 
diverse meanings among institution- ■ 
alized adults, it is not surprising 
that clients and attendants were not 
in consistently close agreement. For 
example, in response to the multiple 
choice question, 18.8% of residents 
said they had no friends while their 
attendant said they had five or 
more friends;, another 21.9% said 
they had "a lot" of friends while 
the attendant was reporting four 
or fewer friends. For the open- 
ended question, 17.4% of comparisons 
involved clients saying they had 
one to three friends, while the 
attendant reported over ten friends. 
Another 17.4% of comparisons in- 
volved clients saying they had no 
friends while the attendant said 
that they had at least one friend. 

Conclusions 

In short, the evidence su7qests 
that the use of quantitative con- 
cepts by our respondents was very 
inconsistent, that the meanings 
of these terms apparently differed 
greatly from person to person. 
In view of this fact, the disappoint- 
ing results achieved using quanti- 
tative multiple choice questions 
are not surprising. Sample 8.2 il- 
lustrates .lome of the difficulties 
our interviewees had with quanti- 
tative concepts. 

ALTERNATIVE .MULTIPLE CHOICE FORMATS 

Quantitative multiple choice 
questions were included in our 
interviews largely in an attempt 
to get an idea of the extent of 
involvement in various activities 
on* the part of our respondents. 
However, we were from the beginning 
concerned that four response alter- 
natives might tend to confuse 
mentally retarded individuals. In 
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A Sampling of Problems with Quantitative Concepts 

1. Q: How often do you leave here to go see your family: A lotj 

sometimes, not much, or never? 
A: ^Respondent said yes as each alternative was given. ^) 

2. Q: How many people sleep in your bedroom? 
A: Charlotte 

3. Q: How many people sleep in your bedroom? 
A: We got a whole bunch of beds bWck there. I can't count 

/ 

'em all. Not righ-. now / 

/ 

4. Q: Are yo\i usually happy or sad? j 
A: I'm always happy sometimes. 

5. Q: Are you usually sadV 
A: Yes, when somebody hits me I am. (On repetitic^ii of the 

question, he said, "No, not very much.") / 

■ 

6.. Q: How much rlo you watch TV - a lot, some, not much or never? 
A: Some, much. 
Q: ' (repeated) 
A: Never. 

^7. Q: Do you watch TV a lot, seine or not much? 
A: Some, not a lot, much. 
A: (repeated) 

A: I'm going to watch Elvis Presley tonight. 



8. Q: How much do you read books, magazines, or newspapers - 

Never, not much, some, or a lot? 
A: I can't read. I'm sorry, but I can't. 
Q: (Repeated since none of the four options was chosen) 
A: A lots. 

And finally, a reconsideration of knee-jerk acquiescence: 

9. Q: Could you buy a new TV with $10.00? 

\ A: Yeah, I do. No, I can't. You can't buy a TV with that 
money. You pulling my leg off. 
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Table 8.2; Yes-No Plus Three-Choice Multiple Choice vs. Four-Choice Multiple Choice 



Content 



How often do you leave here to go see your 
family? 



Four-Choicp 

Multiple Choice 

Jl 



J— 
% RESP f % AGREE 



78.0vf 



How often does anybody in your family come 
to see you? 



How much do you watch TV? 



How often do you go to church? 



How often do yoa go to stores? 



75'. 6% 



How much do you listen to the radio or. record 
player? 



Hnw often do you go out to th. movies? 



How much do you read books, magazines or 
newspapers? 

How often /io you go out tc eat? 



Average: 



55.2% 

49.1% 
41.3% 



49.1% 



52.6% 



.55.2% 



50.0% 



30.0% 



22^., 
70.4%- 
NA 

45.8% 



5.9% 
18.1% 
16.1% 



56.0% 



32.3% 



ies-No s Three Choice 
Multiple Choice 



% RESP % AGREE 



65.7% 



69.0% 



'5^r9% 



51.4% 



60.3% 



55.2% 



50.0% 
48.3% 



57.6% 



52.4% 



22.4% 



27.3% 



70.6% 



15.0% 



42.5% 



NA 

32 0% 



37.5% 



Consistency 



% Consistent 



77.3% 



43.0% 



44.4% 
85.2% 
48.0% 
52.2% 



29.2% 
50.0% 
33.3* 



52.0% 
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an effort' to simplify the task 
demnds but still obtain quantita- 
tive information^ we combined some 
yes-no questions with three-choice 
multiple choice questions asked 
of only those respondents , who 
answered the yes-no question "y^s." 
For example , we asked "Do you leave 
here to go see your family? (if 
yes) Do you go jee them a lot^ 
sometimes^ or not much? The 
comparisons that follow ompare 
the four-choice multiple choice 
format with the yes-no followed by 
contingent three-choice multiple 
choice format. In the analysis 
that follows f responsiveness figures 
have been transformed so that re- 
sponsiveness for the two formats 
can be directly compared^ and con- 
sistency and agreement figures have 
been similarly adjusted (with a 
"no" response to the yes-no question 
coded as a "never" response to the 
four-j-choice multiple choice ques- 
tion) . 

Illustrative Comparison 

One comparison of these two 
formats consisted of asking insti- 
tutionalized adult- ^ both "How much 
do you read be igazines, or 

newspapers? Ne not much, some, 
or a lot?" and "Do you read books, 
magazines, or newspapers? (if yes) 
Do you read books, magazines, or 
newspapv^rs a lot, some or not much?" 

A total of 48.3% of respondents 
responded appropriately to the yes- 
no question and subsequent three- 
rho ice multiple choice question. 
This is slightly less than the 
52.6% who gave appropriate answers 
to the four -choice question. 

Of 26 subjects who respjjfced ap- 
propriately to both formats, only 
50.0% gave consistent answers. The 
pattern of incr ntency is of 
interest: i^^ ^oi ^e to both 
questions, the ^. ..ojjwnbe alternative 
most frequently chosen was the last- 



mentioned. Thus, 11.5% of compar- \ 
isons were inconsistencies due to : 
the client choosing the last option \ 
in both cases. This suggests that 
consistency was reduced in part by 
a last option response bias. In 
addition, 80.8% of clients re- 
sponded "yes" to the yes-no ques- 
tion^ indicating that they at least 
sometimes read. In response to the 
four-choice multiple choice ques- 
tion, only 69.3% of clients in- 
dicated that they at least some- 
times read. This contrast reflects 
acquiescence in response to the 
yes-no foirmat. 

The four-choice multiple choice 
question produced only 18.1% agree- 
ment with attendants. Most dis- 
agreements (50.5% of comparisons) 
occurred because the client re- 
sponded "a lot" (the last option) 
and the attendant said something 
less. Agreement for the yes-no plus 
three-choice multiple choice ques- 
tion was slightly higher at 32.0%. 
Of all comparisons with attendants, 
24.0% were "not much" responses 
from the client ( the last option), 
coupled with a response of either 
"never" or "'^ome" by the attendamt. 
In short, "'ent is low for both 

formats, witn ti. Pattern of in- 
consistencies indicative of some 
last option response bias on the 
part of the residents. 

Additional Comparison 

A total of nine |Comparisons 
of four-choice multiple choice ques- 
tions with yes"r^c^ pljus three-choice 
multiple choice questions were con- 
ducted. Table 8.2 r^epicts respon- 
siveness, consistency, and agreement 
data for-all- nine comparison^. Al- 
though the specific comparison dis- 
cussed above is generally repre- 
sentative Lheise sets of ques- 
tions, j-t ""e-^r that there is a 
lot of variability within the set. 
Nevertheless, the genere.! conclu- 
sions suggested by the "read" ques- 
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tions are linaltered; specif ically, 
neither format appears attractive. 
Responsiveness, consistency , and 
agreement are poor for both formats, 
neither appears to offer a definite 
advantage over the other. Agreement in 
particular was poor for both formats 
37.5% for the yes-no plus follow-up 
approach versus 32.3% for the four- 
choice questions. Only the questions 
about church attendance appeared to 
work well, possibly because there 
is a relatively clear external 
standard suggesting that once a 
week or more is "a lot." 

It should be pointed out that the 
last option response bias discussed 
above' was observed frequently but 
not inevitably within this set. 
It appears that mentally retarded 
respondents are often biased in 
favor of the last-mentioned al- 
ternative , /but that this set is 
weak enmlgh that it is frequently 
overshadowed by other factors. The 
important point is that these 
quantitative multiple choice ques- 
tions do not yield useful informa- 
tion. 

VERBAL MULTIPLE CHOICE QUESTIONS VS. 
MULTIPLE CHOICE QUESTIONS 
WITH PICTURES 

A direct comparison of standard 
multiple choice questions with 
multiple choice questions offering 
pictures as response alternatives 
was undertaken to see if one format 
had a definite advantage. The use 
of pictures as response alternatives 
was primarily intended to elicit 
information from clients with few 
verbal skills. These comparisons 
involved subjective questions. 

Illustrative Comparison 

Institutionalized adults were 
asked, "How do you like living here: 
a lot, some, not much, or not' at 
all?" and "Which picture shows how 



you feel about living here? Point 
to the picture." (Sad face followed 
by somewhat sad, somewhat happy, 
and happy faces,) 

Consistent with prior observations 
regarding responsiveness to picture 
formats, the use of pictures in- 
creased responsiveness. In response - 
to the picture alternatives, 70.7% 
of clients responded appropriately 
by pointing; for the verbal multiple 
choice question, responsiveness was 
50.9%. 

Responding across the two formats 
of the multiple choice question 
was very inconsistent; 'the overall 
consistency figure was only 21.4%. 
As to the pattern of this incon- 
sistency, it appears that the use 
of pictures tended to spread re- 
sponses across response alternatives. 
Responding to the .verbal format 
was biased toward the "a lot" end 
of the continuum, with 35.7% of 
clients replying "a lot" (the first 
option) . Responding to the pic- 
ture format did not favor one end 
of the response continuum, as 
32.1% of residents answered "a lot" 
and 39.3% replied "not at all," 
This could reflect a first option 
response bias in the case of 
verbal alternatives, or simply a 
greater tendency toward random re- 
sponding in the case of picture 
alternatives. Since a bias toward 
choosing the last option on mul- 
tiple choice questions has been 
observed elsewhere, we favor the 
latter interpretation. 

Although these questions were s\ab- 
jective, we asked attendants to 
characterize residents ' satisfaction 
with the institution. Agreement for 
the picture format was 21.3%. 
Agreement for- the word format was 
slightly higher, at 32.3%. In both 
cases, inconsistencies arose pri- 
marily because attendants rated 
clients as having more pos^itive.j; 
feelings than were reported by 
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the clients themselves. This may 
well indicate that there is a real 
difference of opinion between 
clients and attendants on the issue 
of satisfaction with the institu- 
tion. However r con j.^t^ :y data 
still suggest that neither of these 
formats successfully elicited valid 
answers, and that the picture formats 
while it enabled many persons to 
respond, is particularly suspect'. 

Additional Comparisons 

Two additional comparisons of 
picture and verbal multiple. choice 
questions were conducted, but with- 
out data on agreement with attend- 
ants. These questions involved 
satisfaction with the food and 
with the people at the institution. 
In both additional comparisons, 
the use of pictures was associated 
with higher responsiveness than was 
the verbal questions. These com- 
parisons were 67.9% vs. 50.0%, 
and 67.9% vs. 55.2%,' respectively. 
Consistency figures for the questions 
regarding food and people were 33\3% 
and 37.8%, respectively. This very 
low consistency, indicative of a 
lack of reliability of responses 
to these quantitative multiple 
choice questions, suggests that 
neither format successfully gen- 
erated reliable information. 

Conclusions 

Clearly, the use of pictures as 
r*^,sponse alternatives increased re- 
sponsiveness to these multiple choice 
questions. However, consistency 
was so low as to suggest that neither 
format is very useful. Based on 
a^eement data and the pattern of 
inconsistencies obe^rved, it appears 
that the gain^in responsiveness 
achieved by using pictures is not 
accompanied by an increase in re- 
liability and validity. Sample 8.3 
illustrates some of the problems 
with these picture choice questions. 



These discouraging data are not 
surprising in view of the fact 
(discussed in' Chapter 6) that test- 
retest reliability for one crr-ny-ir^ 
of picture multiple choice 
tions averaged only 46.2%. 
can be compared roughly witii r ,j'< 
reliability for verbal multiple 
choice questions on different 
topics, as reported in Chapter 6.) 
Responses of mentally retarded 
persons to quantitative multiple 
choice questions, whether verbal 
or pictorial, appear to be in- 
herently unreliable . 

DISCRETE MULTIPLE CHOICE QUESTIONS 

The multiple choice questions 
previously discussed, which appear 
unsatisfactory for use wi^h JJ^®^ 
tally retarded populations f T-n- 
volved subjective judgments on 
the part of respondents and used 
response alternatives which differed 
along a quantitative dimension. 
Another group of multiple choice 
questions in our interviews dealt 
with factual information and offered 
response alternatives which differed 

in a qualitative manner — that is, 

discrete response optior'^. 

Discrete multiple choice questions 

in both "point to the pictu>"^." and 

verbal response fom.. 

pared directly with opea-enuea 

questions soliciting the same 

information. 

Illustrative Comparison 

Community children were asked 
how they traveled to school using 
each of the following three ap- 
proaches: "Here are some ways 
people get to school. (Pointing. to 
pictures). They take the bus, 
somebody drives t ^ t -.chod in 
a car, they wal.., - r ' to 

school on their bicycles. Which 
way do you get to school most days? 
point to the picture."; "Most 
days, how do you get to school?"; 
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,^ Rc"'-'^'"'^ with Picture Choice Questions 
Failure Sr.ory : 

Q: Which picture shows how you feel about living here? 

Point to the picture. 
A: (pointed to "not at all" saddest face of four faces) 
Q: Why do you feel that way? 
A: I don't like to feel that way. 
Q: (repeated) 
A: I don' t feel that way. 

Weird Story _ 
Q: Which picture shows how you feel' about the people here? Point 

to the picture. 
A: (did not point) They're not good. They. . .(stopped) 
Q: (repeated) 

A: (did not point) Cause they get mad about the way she acts - she' 

do anything I say (reference not clear) 
Q: Why do you feel that way? 

A: They don't assume _I harp on me to say. They don't \anderstand 
what I say. 
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Success Story ; • * 

Q: vVhich picture shows how you feel about the people here. 

Point to the picture. 
A (pointed to "not at all" saddest face) 
Q: Why uo you feel that way? 

A: He looks like he' s crying, tears falling from his oyoi. . 

People were /making fun of him. He wants to go home to his parents 
Note: Unfortiinately this success story is the exception. 
Typically respondents had difficulty giving consistent responses 
to these picture choice questions about satisfaction. 



\ 
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and "Most days, do you to 
school on the bus, in a ^car, by 
walking-, or riding a bicycle?*' 

Responsiveness to the picture mul- 
tiple choice question was 100%; all 
children asked the question were 
able to I'^-ispond appropriately by 
point ino to one of the pictures . 
For the verbal multiple choice 
question, respor-,iveness was 94.5%. 
.The- open-ended question elicited 
the lowest percentage oi responses 
from children (73 •6%). Clearly, 
the use of multiple choice que ms 
in general and of picture mul 
choice in particular optimize:., 
sponsiveness • 

Consistency for comparisons using 
the discrete multiple choice format 
was exceptionally high. Comparison 
of the picture multiple choice 
format with the open-ended question 
produced a consistency figure of 
95.2% for 41 response pairs. In 
the comparisons of the verbal mul- 
tiple choice and open-ended ques- 
tions, 95.1% of children responded 
consistently. The comp^.rison be- 
tween the. two multiple choice ques- 
tions produced a consistency figure 
of 86.9%. For the latter two 
comparisons, most inconsistencies 
arose because children often 
chose the last option in the verbal 
multiple choice format/ again sug- 
gesting a last option bias. How- 
ever, this interpretation must be 
tempered by the realization that 
consistency was so high that pat- 
terns of incr:isistency involved 
very few subjects? the major 
conclusion remains that consistency 
was remarkably high for these ques- 
tions. 

Parent-child- agreement figures for 
the picture multiple choice (85.5%) 
and the open-ended question (85.4%) 
were almost the same. There was 
slightly less agreement between 
parents and children on the verbal 



multiple choice format (77.4%) • The- 
only pattern observed in the disa- 
greements was a slight last option 
bias on the verbal multiple choice 
question; 13.2% of pair^ disagreed 
because the child said "bike" (the 
last option) while the parents said 
either "car" or "bus." In this 
case, then, the picture multiple 
choice question is preferable on 
grounds of both responsiveness and 
validity. 

Addi^ Lonal Comparisons 

itiere was another comparison / 
also involving community children, 
of four question^ ^skj^^q in what 
kind of place the child lived 
(house, apartment building, trailer 
house, or duplex). The questions 
were discrete multiple choice, pic- 
ture and verbal, with response al- 
ternatives presented in original 
and alternative orders as a test 
for order effects. 

Average responsiveness for the pic- 
ture choice questions was 96,5%, 
as opposed to 81.8% for the verbal 
questions. Consistency of response 
between the two verbal choice formats 
was slightly higher than that for 
.^^ the picture questions — 85.7% as 
' compared to 78.2%. For the two 

comparisons between word and picture / 
formats presenting options in the 
same order, consistency averaged 
81.8%. The two comparisons be- 
tween verbal and picture formats , 
V7hich presented options in different 
orders yielded a similar average con- 
sistency figure of 78.8%. Thes^e 
consistency data suggest that (1) 
the superiority of the picture ques- 
tions observed earlier was not sup- 
ported, and (2) order biases are not 
particularly important in these for- 
mats, since different order compari- 
sons are about as consistent as same 
order comparisons. Average agreement 
between parents and their children 
for the verbal choice questions was 



81. 8% f while average agreement for 
the picture format questions was 
70,5%. This again reverses the 
conclusion drawn from the transpor- 
tation questions^ but it is not clear 
why the validity of responses to the 
picture choice question was lower 
here . 

Conclusions 

One conclusion which can be drawn 
from these comparisons' is that dis- 
crete multiple choice questions about 
basic facts differ strikingly, from 
their quantitative counterparts. 
These questions yielded high respon- 
siveness^ high consistency^ and 
high agreement relative to most other 
formats. Comparisons between pic- 
ture choice and verbal choice ap- 
proaches resulted in inconsistent 
findings, so that a global judgment 
of their relative merits it impos- 
sible. There is some evidence for 
a last option bias in verbal mul- 
tiple choice questions, but thir. 
phenomenon is unstable. To the 
extent that order bias affects 
verbal choice questions, it at 
least does not consistently reduce 
their validity below that of picture 
choice questions. However, since 
picture multiple choice ques+,ions 
consistently maximize responsiveness 
and there is no reason to bf;lieve 
that they are consistently less 
valid than verbal choice questions, 
picture multiple choice would appear 
to be an excellent method for elic- 
iting factual information from 
mentally retarded respondents. Un- 
fortunately, we did not test the 
discrete multiple choi(^^- format with 
other content areas, so it is im- 
possible to determine whether the 
success found here would generalize 
to other topic areas, or whether re- 
tarded childrer simply have solid 
enough vinderstanding of how they get 
.to school and whore they live that 
almost any format would work. 



Ye&-No Vs. Open-Ended Questions 

LISTING QUESTIONS 

One type of information which 
we soug'rc'to obtain £rom mentally 
retarded persons was eniamerations 
of activities, problems, interests, 
and so on. The decision faced by 
an interviewer iiiuerested in ob- 
taining such information hinges 
on the issue of question structure. 
Should one choose highly structured 
formats and run the risk of re- 
sponse biases such as acquiescence 
or should one choose open-ended 
formats and run the risk that few 
interviewees will be able to an- 
swer the questions? To evaluate 
this problem, we directly compared 
open-ended questions asking for 
active enumeration of responses 
with sets of yes-no questions in- 
tended to obtain the same informa- 
tion . 

/ 

Illustrative Comparison / 

One comparison of these two 
formats, drawn' from the institu- / 
tionalized children sample, ac- / 
curately represents finding for ^the 
group. of comparisons as a whole. 
Institutionalized children were 
asked eight yes-no questions/about 
sports in which they might /par- 
ticipate (for example, "Dp you 
play football?"). Residents were 
also asked an open-ended question 
soliciting the same information: 
"Do you play any sports?" (if Y^s) 
Which ones?" Respo.ises tc the 
open-ended format were s\jbsequently 
coded into categories correspond- 
ing, to the yes-no questions, so 
that each client responded "yes" 
or "no" when asked if he or she 
plays a given sport and was cate- 
gorized as -having mentioned or not 
mentioned that sport in response to 
the open-ended question. Attendants 
were asked a parallel list of yes- 
no questions ("Does this client play 
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any of these sports: Football? - 
Baseball? and so on). 

Average responsiveness to the eight ^ 
yes-no questiojis was 85.6%. This 
compares favoi/ably with the 55.2% 
figure for thk open-ended question. 
Consistency was evaluated by aver- 
aging consistency across the eight 
pairings of/ a yes-no question with 
its corresTOnding response category. 
Average consistency was 49.7%, with 
inconsistency resulting almost ex- 
clusively from pairings in which 
the client replied "yes" on the yes- 
no format but failed to mention that 
response category in response to 
the op4n-ended question. Of all 
comparisons, fully 48.1% were incon- 
sistencies of this type. As well 
as i4lustrating the ubiquitous im- 
pact of acquiescence, this finding 
highlights the possibility that 
underreporting is a serious problem 
iti the- response of mentally retarded 
^persons to open-ended questions. 
/Examination of data regarding agree- 
/ ment with attendants is necessary to 
determine which effect is stronger. 

Client-attendant agreement was cal- 
culated for the eight-yes-no ques- 
tions and for the eight categories 
of response to the open-ended ques- 
tions. Aver:;ge agreement for the 
yes-no ques-cions was 52.1%, as op- 
posed to 60.1% for the open-ended 
question. For the yes-no questions, 
42.3% of all pairings were dis- 
agreements in which the child said 
"yes" while the attendant said 
"no," and an additional 5.6% were 
disagreements in which the child 
said "no" and the attendant 
"yes." Again, the overwhelming im- 
pact of acquiescence is evident. 
For the open-ended question, 31.9% 
of client-attendant pairs disagreed 
in that the attendant said "yes" 
regarding a sport not mentioned by 
the retarded respondent. Only 8.1% 
of pairings were disagreeHi^.its in 



which the child mentioned a sport 
not indicated by the attendant. 
Thus, retarded interviewees tendea 
to underreport their activities 
in response to the open-ended ques- 
tion. In short, acquiescence leads 
to overreporting in response to 
yes-no lists and clients are prone 
to xanderreport in response v ' open- 
ended questions; the latter bias 
may be slightly less dramatic. 

Additional Comparisons 

Four additional sets of ques- 
tions, asked of institutionalized 
adults and institutionalized chil- 
dren, compared open-ended and yes- 
no listing^ questions. These sets 
of questions solicited information 
about th4 client's problems, actions 
of others that bother the client, 
nice tilings people do for the client, 
and participation in arts and 
crafts. 

In each comparison; far more clients 
gave appropriate responses to the 
yes-no questions than the open-ended 
questions. The responsiveness 
figures for the four yes-no ques- 
tionf' ranged approximately from 
60-805^, while those for the open- 
ended questions ranged from 20-50%. 
The superiority of yes-no questions 
in eliciting appropriate responses 
is clear. 

As with responsiveness, the consist- 
ency data for these comparisons 
are remarkably uniform and echo the 
results from the illustrative com- 
parison. That is, the two formats 
generate very different pictures of 
our clients, because clients claim 
far more activities, problems, and 
so on sin response to the yes-no 
format than when asked open-ended 
questions. Consistency for these 
comparisons varied, but in all 
cases the largest source of inconsist- 
encies consisted of "yes"' responses 
to the yes-no format coupled with 
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"not mentioned" responses to the 
open-ended question. 

Only one of these comparisoils , 
contrasting an open-ended-ques- 
tion asking the client to enumerate 
problems that he or she has with ^ 
a series of yes-no questions about 
particular problems, included 
attendants' responses. Agreement 
was far lower for the yes-no than 
the open-ended questions (average 
agreement of 45.6% and 92.4%, 
respectively) . The yes-no format 
again led to. overreporting because 
of acquiescence, as 44.9% of com- 
parisons were disagreements in 
which the client answered "yes" 
and the attendant answered "no." 
The fact that clients clearly 
acquiesced even in response to these 
questions, for which "yes" responses 
were not socially desirable responses, 
indicates that in at least some 
cases acquiescence per se can over- 
rule the influence of social de- 
sirability. The open-ended format 
again appeared to lead to under- 
reporting, as all disagreements 
(7.5% of pairs) occurred when an 
attendant said "yes", regarding a 
problem not mentioned by the client. 
The very high agreement on this 
question resulted from the fact that 
neither clients noir attendants men- 
tioned many problems; in 89.1% 
of pairings, the two agreed in that 
neither indicated a given problem. 

Conclusions 

These comparisons are an excel- 
lent illustration of the' difficul- 
ties of interviewing mentally re- 
tarded individuals, as the use of 
two different methodologies to 
seek the same information yielded 
completely dissimilar pictures of 
the target population. Yes-no ques- 
ticas have the advantage of high 
responsiveness, but are clearly sub- 
ject to strong systematic bias 
(acquiescence) . . Low agreement for 



these questions suggests that 
acquiescence leads mentally re- 
tarded persons to overreport what- 
ever they are being asked about. 
Open-ended questions have the ad- 
vanta ' of relatively high agree- 
men-*:, «=:nerate very low respon- 

sivent;,. , that information 
drawn from open-ended questions re- 
presents only a selected fraction 
of the population. Furtlfier, the 
number of items enumerated in re- 
sponse to these questions is low 
even among those who respond ap- 
propriately; the very low pro- 
portion of mention-mention agree- 
ments in these comparisons raises 
the possibility of underreporting-" 
by both mentally retarded resj5bn- 
dents and their attendarit-s. Ihc 
truth probably lies somewhere be- 
tween these extr:eities of under- 
reporting and- "overreporting. Un- 
fortunately, it se^ms impossible - 
to establish exactly^.where, thi^ugh 
our 'Indings hint that overreporting 
in response to yes-no questions may 
be the more pron6unced bias. 

DECISION-MAKING QUESTIONS 
Illustrative Comparison 

Two open-ended questions, "Who 
decides v;hat chores you do?"\ and 
"Who decides what chores to do?" 
were compared with a yes-no ques- 
tion, "Do you decids what chores to 
do?" The first open-ended ques- 
tion was asked on both forms, al- 
lowing for three comparisons of 
the open-ended and yes-no formats. 
These questions differ fundamentally 
from open-ended questions previ- 
ously discussed, in tliat they do , 
not solicit enumerations. Rather^ 
the client's response was coded 
into one of two categories (client 
decides vs. someone else decides). 
Thus, the information demanded by 
these questions is closely compar- 
able to that solicited by either- 
or questions. 
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Responsiveness for the three open- 
ended questions ranged from 32.8% 
to 34.5%, averaging 33.9%; This 
L'igure was clearly exceeded by the 
54.4% responsiveness for the yes- 
no question; the superiority of the 
yes-no format in eliciting appropri- 
ate responses emerges with monotonous 
regularity. 

Consistency for the three comparisons 
of the yes-no questions with open- 
ended questions ranged from 56.3% 
to 68.9%, averagin»j 63.9%. Exam- 
ination of the data reveals a def- 
inite pattern in the inconsistencies. 
Of all comparisons, 32.4% were errors 
in which the client responded "some- 
one else" to the open-ended question, 
but answered "yes" (client decides) 
when asked the yes-no form of the 
question. Tliis fraction accoiints 
for nearly all the inconsistencies; 
clearly, acquiescence powerfully in- 
fluenced the responses of clients 
to the yes-no format. As a result, 
one would conclude on the basis of 
the yes-no question that 52.1% of 
the clients decide for themselves, 
whereas:' the three open-ended ques- 
tions yield a much lower figure 
(23.4%). 

Agreement with attendants differed 
radically for the yes-no and open- 
. ended questions; agreement for the 
yes-no question was. 34.6%, while 
agreement averaged 77.9% for the 
three open-ended questions. For 
the yes-no question, 65.4% of com- 
parisons were disagreements in which 
the client responded "yes" (client 
decides), while the attendant re- 
sponded that someone else decided. 
Again, acquiescence operated to re- 
ducfa the validity of responses of 
retaroed adults to the yes-no ques- 
tion. 

Additional Comparison 

A similar comparison of yes^no 
and open-ended questions, also 
conducted with institutionalized 



adults, involved decisions about 
spending money. The results of 
this comparison are virtually iden- 
tical to those of the previously 
discussed comparison. 

Conclusions 

As always, responsiveness for 
yes-no questions was far higher 
than for open-ended questions. 
However, in these comparisons the 
reliability and validity of the 
open-ended questions weije superior. 
Client-attendant agreement was in 
each case much higher for the open- 
ended questions, as acquiescence 
apparently reduced the validity of 
responses to the yes-no questions. 
It appears that open-ended ques- 
tions soliciting information which 
can be coded into a few present 
categories may be quite useful, as 
long as one is willing to make a 
sacrifice in responsiveness. 

Yes-No Vs. Either-Or Questior^s 

On another test of alternative 
formats, we compared, sets of yes- 
no and either-or questions dealing 
with the same topics. A clear 
comparison of these formats re- 
quires an ur;derstanding of the 
strengths and weaknesses of each 
questioning strategy taken sep- 
arately. Thus', we begin by analyz- 
ing responses to sets of oppositely 
worded yes-no questions and to sets 
of either-or questions with response 
alternatives presented in reverse 
order. 

OPPOSITELY WORDED YES-NO QUESTIONS 
Two comparisons of oppositely 
worded yes-no questions will be 
discussed below. Rfesponsiveness 
is unaffected by reversincj the 
wording of yes-no questions and 
is not analyzed here. 




Table 8-3: Oppositely Worded Yes-No Questions 



/^jes somebody else decide 
how you spend your money? 


bo you decide how 
to spepd your money? 






No 


Yes 


Totrl 




No 


11.8% 


26.5% 


39.3% 




Yes 


11.8% 


50.0% 


61.8% 




Total 


23.6% 


76.5% / 










Consistency 


= 38.3% 





Illustrative Comparison 

Institutionalized adults were 
asked "Do you decide how to spend 
your money?" and "Does somebody 
• else decide how to spend your 
money? The contingency table 
for comparison of the two questions 
is depicted in Table 8.3. Only 
38.3% of the adults^ far below the 
50% which would be expected if re- 
sponding to both questions were 
random, gave consistent responses. 
Whi'.e 76.5% said 'Ves" to "Do you , 
decide...?" 61.8% also said "yes" 
to "Does somebody else decide...?" 
Fully 50.0% of the response com- 
parisons were self-contradictions 
in which the client responded "yes" 
to both forms of the auestion. . 
Acquiescence is clearly a major de- 
terminant of responses to these' 
questions . 

For the "Do you decide...?" ques- 
tion, agreement with' attendants was 
55.5%. For the /oppositely worded 
question, agreement w^ 54.3%. In 
both cases, the majority of dis- 
agreements consisted of a "yes" re- 
sponse from the client, paired with 
a "no" response by the attendant, 
again indicating the importance of 
acquiescence by mentally retarded per- 
sons. Another set of oppositely- 



worded yes-no questions, asked of 
community children, produced vir- \ 
tually identical findi^ngs. 

Conclusions 

Both of these pairs of oppositely 
worded yes-no questions clearly il- 
lustrate the enormous importance of 
acquiescence among mentally retarded 
persons. These respondents tend to 
answer "yes," even when such re- 
sponses are self-contradictory. 
Additionally, clients tended to 
respond "^es" far more often. than 
significant others. Clearly,, 
acquiescence effects are powe^ul 
enough that the validity of reA^ 
tarded persons* responses to yes- 
no questions is minimal. ^ 

REVERSED EITHER-OR QUESTIONS 
Illustrative Comparison 

A nijmber of pairs of either-or 
questions were asked, such that 
the members of the pair dealt with 
identical content but the order of 
presentation of response alterna- 
tives was reversed. One such com- 
parison in the children's community 
sample will be discussed below. 
The questions were "Most days, does 
somebody else tell you what to 



8.26 , ;^ 

17 j \ 



wear to school r or do you decide?" 
and "Most days , is it up to you 
to decide what to wear to school or 
does somebody else tell you wha"'^ 
to wear?" 

Responsiveness, 63.6% to the first 
either-or question and 65.5% 
to the second, was unaffected by 
the change in wording. Consistency 
was impressive for this pair of 
questions, in that 83.3% of chil- 
dren who responded appropriately 
to both questions responded in a 
consistent manner. The type of 
inconsistency was evenly divided, 
suggesting a lack of position 
preference in response to this 
question . 

Agreement with parents was fairly 

high for both forms of the question. 

Agreement was 72.5% for the ques- 
\ tion in which "somebody else" was 

"first alternative, and was 73.2% 
for the other question. For the 
former question, types of dis- 
agreement were evenly mixed; for 
the latter, most inconsistencies 
(19.5% of comparisons) involved 
the choice of the last option by 
the child. This constitutes evi- 
dence for a weak ppsition bias in 
favor of the last-mentioned alter- 
native. 

Conclusions 

This pair of either-or questions 
generated relatively high consist- 
ency and parent-ckild agreemeint. 
There was some evidence of a weak 
position bias by mentally retarded 
persons, but little indication of 
powerful response^ef fects. To this 
point, then, either-or questions ap- 
pear superior to yes-no questions, 
but l.ead-to-head comparisons of 
/ the two formats are needed before 
we can be sure. 



YES-NO VS. EITHER-OR: FOUR-QUESTION 
SETS FROM INSTITUTIONALIZED ADULTS 
Two sets of questions asked of 
institutionalizecl adults, com- 
paring either-or and yes-no ques- 
tions, were repeated in two inter- 
views a week apart, yielding a 
measure of test-retest reliability 
as well as consistency m^\asures. 

Illustrative Comparison 

One of these four-question sets 
inquired about the client's happi- 
ness, using the following questions: 

Are you usually happy? 

Are you usually sad? 

Are you usually happy or sad? 

Are you usually sad or happy? 

All questions were repeated on 
hotU administrations of the inter- 
view, so that a total of eight 
questions is involved in this com- 
parison. 

Average responsiveness for the eight 
questions was 65.5%. Responsiveness 
ranged from 58.6% to 70.7%, and 
was slightly higher for the four yes- 
no questions than for the either-or 
questions — 68.1% vs. 62.8%. 

One measure of consistency compared 
each question with its identical 
counterpart on the other form, giving 
a measure of test-retest reliability. 
For the "happy" and "sad" yes-no 
questions respectively, consistency 
figures were 97.3% and 60.5%. For 
the two either-or questions, consist- 
ency figures were 87.9% and 94.1%. 
resulting in average consistency 
for* the four comparisons of 85.0%. 

One point of interest is that consist- 
ency was very discrepant for the 
two yes-no questions. It seems likely 
^:hat this is an effect of acquies- 
cence; fcr the "happy" wording, clients* 
"true" r.>sponse and their response 
set operai^ed together, so that "yes" 
responses were firmly established. 
On the "sad" wording, response bias 
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and the "true" response operated 
in opposite directions. This conflict 
apparently introduced a degree of 
uncertainty which caused some clients 
to change their responses from inter- 
view to interview. Thus, the test- 
retest reliability of the yes-^no 
questions appears to have depended 
in part on the wording of the ques- 
tions. In contrast, consistency 
for the either-or questions dif- 
fered Dnly slightly as a function 
of wording. Test-retest reliability 
for both these types of questions 
was reasonably high. It was some- 
what higher for either-or questions 
(on the average, 91.0% vs. 78.9%), 
probably because response set was 
of less importance for the either- 
or format. 

Another type of consistency involved 
comparison of oppositely worded 
questions within a given format. 
For the four comparisons of opposite- 
ly worded yes-no questions asked 
on the two occasions, consistency 
averaged only 53.1%, ranging from 
46.2% to 55.6%. Clearly, acquiescence 
was a major determinant of response 
for these questions, as on the aver- 
age, 43.9% of clients contradicted 
themselves by responding "yes" to 
both questions. For comparisons of 
oppositely worded either-or questions r 
consistency ranged from 77.8% to 
99.7%, averaging 86.2%. This rela- 
tively high, uniform consistency 
attests to the usefulness of either- 
or questions in interviewing mentally 
retarded individuals. However, there 
was some indication of a last option 
position bias inherent in responses 
to either-or questions. Clients 
'"mded overwhelmingly to respond 
"happy" to these^ questions, but this 
tendency was more pronounced when 
"happy" was the last mentioned re- 
sponse alternative (93.4% reported 
being "happy" when "happy" was the 
last choice vs. 81.8% when it was 
not) . This response bias, however. 



was much less powerful chan was 
the acquipscence in response to 
yes-np questions. 

This set of questions allowed fcr 
sixteen comparisons of yes-no to 
either-or questions. The eight com- 
parisons of the question "Are you 
usually happy?" with either-^r ques- 
tions yielded an average consistency 
figure of 96.7%, ranging from 82.9% 
to 100.0%. These comparisons tended 
to be very consistent because a 
large majority of clients responded 
"yes" to the yes-no question and 
"happy" to the either-or format; 
an average of 8'7.4% of cojuparisons 
were consistencies or x-h'is type. 
However, for the eight comparisons ■ 
of the question "Are you usually 
sad?" with either-or questions, con- 
sistency averagod only 57-3%. For 
this set, an average of 40.0% of 
format comparisons were inconsist- 
encies in which the/ client re- 
sponded "yes" to the yes-no ques- . 
tion but answered, ''happy" when allied 
the either-or qu^s'-ion. Clearly, 
"happy" was geneWlly the "true" 
response of mentally retarded 
individuals to these questions, 
and responses to the either-or ques- , 
t^^ns tended to reflect that fact j 
/viii spite of some order preference) . 
However, when the yes-no question 
was worded so as to bring tendencies 
toward acquiescence and the "true" 
response into conflict, acq?iiescence 
caused many clients to inaccurately 
report tTiat they are usually sad. 
The striking difference between 
the 87.4% and 57.3% average consist- 
ency figures for comparisons of 
either-or questions with the two 
yes-no questions attests to that 
fact. 

Average agreement with attendants 
for the either-or questions was 
85.8% and ranged from 79.3% to 94.2.. 
Average agreement for the yes-no 
questions was somewhat lower at 72.1%. 
This reduction in agreement is ac- 
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counted for by the very low agree- 
ment for the yes-no question "Are 
you usually sad?" which was 50.0% 
and 50.7% on the two forms. For 
this question, an average of 43.7% 
of client-attendant pairs were 
inconsistent in that the client 
said "yes" fsad) and the attendant 
indicated that the client was usu- 
ally h^ipPy- Again, the major con- 
clusion ^is that acquiescence largely 
invalidates the responses of men- 
tally retarded respondents to yes- 
no questions. 

\ • 

Additional Comparison 

Another comparison of yes-no 
and either-or questions, including 
measures of test-retest reliability, 
was conducted v/ith institutionalized 
adults. This s^t of questions asked 
whether clients usually spend their 
time alone or .with ptJiex-people . , . 
Average responsiveness .for the 
eight questions was. 63. 2%, and was 
slightly superior for the yes-no 
questions (68.3% vs. 58.0%). Test- 
retest reliability for the set av- 
eraged 76.2%. As ill the previous 
set, the reliability of yes-no ques- 
tions was augmented when acquies- 
cence set and the clients* "true" 
answer coincided and was diminished 
when they were at odds. 

Response consistency was again abys- 
mally low when the oppositely worded 
yes-no questions were compared, av- 
eraging 52.0%. The most frequent 
response combination (43.4% of com- 
parisons) had-clients contradicting 
themselves by responding "yes" to 
both questions. Comparisons^ between 
oppositely worded either-or ques- 
tions averaged 75.1%, suggesting 
superior validity^for either-or 
questions- Again d l^st option re- 
sponse bias was observed, as the 
• most frequent inconsistent response 
combination (18.3% of all compari- 
sons) involved the client choosing 
the last option both times / as op- 



posed to only 6.6% of clients who 
chose the first option both times. 

Comparisons of yes-no and either- 
or questions did not ir^eveal the rad- 
ical differences in consistency fig- 
ures for the two yet'-no questions 
that characterized the "happy-sad" 
set. For the eight comparisons of 
either-or questions with "Are you 
usually with other people?" con- 
sistency averaged 69.5%. For the 
question "Are you usually by your- 
self?" consistency^ averaged 68.7%. 
This probably reflects the fact that 
answers were much more balanced 
between the two response alternatives 
for the alone-with others questions 
than foir^ the happy-sad questions. 

Average agreement for \:his set was 
60.8% for the yes-no questions and 
60.3% for the either-or questions. 
Again, agreement figures for the 
two yes'-no~quest4o7iS -were discrepant, 
averaging 51.4% for "Are you usually 
by yourself?" and averaging 70.2% 
for "Are you usually with other 
people?" For the either-or questions, 
agreement was slightly higher for 
the question for which the most 
frequently chosen response alterna- 
tive was the last ment/ioned, 63.5% 
as opposed to 55.2%. T^hus , client-/ 
attendant aqreement is an over- / 
estTmatg^''of^ response ^validity when 
the "true" response and client 
response bias happen to coincide. 
This effect is apparent in the 
either-or comparisons, where agree- 
ment is greater when the "true" 
answer (with others) is the last- 
mentioned alternative. However, 
it is far more notable for the ye3- 
no questions, where agreement is 
much higher when "yes" indicate?; 
that che client is usually with 
other people. 

Conclusions 

Both sets of questions suggest 
that either-or questions compare 
favorably with yes-no questions 
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as a method of gaining infonT^ation 
from mentally retarded persons. 
Both either-or and yes-no questions 
involve some response biases which 
reduce the validity of responses . 
However, the acquiescence activated 
by yes-no questions is a crippling 
deficiency, whereas the. position 
bias inherent in the either-or 
format is merely annoying. In the 
second set of questions discussed, 
the advantage of the either-or 
format is not so great. Neverthe- 
less, consistency for the either- 
or questions in both cases approaches 
the test-retest reliability for 
these questions, and even in the 
second case, either-or questions 
are less affected by response set. 

SUPPLEMENTARY YES-NO 

VS. EITHER-OR COMPARISONS 

The set of yes-no versus either- 
or comparisons described above con- 
stitutes only a fraction of the 
data we collected regarding the . 
relative merits of these two formats. 
Four separate four-question. sets, 
similar to the above examples, were 
included in interviews with insti- 
tutionalized children. In addition, 
two direct comparisons of isolated 
either-or and yes-no .^questions were 
conducted in interviews with insti- 
tutionalized adults. Although some 
variation in patterns of response 
to thes^ two formats was of course 
observed, the consistency with which 
these data characterized yes-no and 
either-or questions was remarkable. 
Specifically, in every case the 
responsiveness data slightly favored 
the yes-no format. However, in every 
comparison the evidence regarding 
validity * unequivocally recommended 
the either-or questions, usually 
be(::ause of the powerful effect of 
acquiescence on the yes-no format. , 
' Again we must conclude that there 
is no point in eliciting invalid 
responses; responsiveness must yield 
to validity as a more essential cri- 



terion. Direct comparisons of yes- 
no and either-or questions suggest 
that the either-or format is pref- 
erable as a means of gaining in- 
formation from mentally retarded 
persons. 

The Use of Pictures With Yes-No 
Questions 

In a continuing effort to maxi- 
mize the amount of information that 
we could gain from clients with low 
verbal skills, we asked some yes- 
no questions which included pictures 
to clarify the question. Two sets 
of questions directly compared yes-no 
questions using pictures with the 
same yes-no questions not using pic- 
tures. These comparisons can tell 
us whether the use of pictures in 
any way serves to reduce the acquies- 
cence so commonly observed in re- 
sponse to yes-no' questions. 

Illustrative Comparison 

The first such set, asked of 
institutionalized children, involved 
indoor games. The children were 
twice asked four yes-no questions 
such as "DO you play checkers?" 
once with words only and OQce in- 
cluding pictures of people engaged 
in the gcime named. 

Responsiveness, 84.4% for the ques- 
tion with words only and 82.2% for 
the words plus pictures format, 
was apparently unaffected by ques- 
tion type. Respondent consistency 
for the four pairs of yes-no ques- 
tions averaged 89.8%. Children 
tended overwhelmingly to respond 
"yes," with the result that most 
comparisons (75.1%) were consist- 
encies in which the child responded 
"yes" both times. 

The agreement data contain the most 
useful information regarding this 
format comparison. These data are 
summarized in Table 8.4. Overall 
agreement averaged only 42.4%. By 
far the greatest fraction of dis- 
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Table 8.4: Yes-No Questions vdth Words Plus Pictures vs. Words Only 



Child-attendant Agreement 





Topic 




•TotaU 


Yes 


. Ho 


Child-Yes; 


Child-No 


yuestion iy|Jc 


H 


Agreement 


Agreement 


Agreement 




U.U. ilCO 




Checkers 


45 


53.3 


42.2 


11.1 


42.2 


4.4 


No pictures 

• 


Cards 


44 


52.3 


50.0 


2.3 








Dominoes 


43 


32.6 




Q 'I 


55 8 


1L6 




Other games 


41 


51.2 


39.0 


12.2 


29.3 


19:5 


Group Average 


43.3 


47.4 


38.6. 


8.7 


42.1 


10.6 








Checkers 


43 


62.8 


44.2 . 


18.6 


34.9 


2.3 


Pictures 










2.4 


38.1 




Cards 


42 


57.2 


54.8 


4.8 




Dominoes 


43 


32.6 


23.3 


9.3 


'55.8 


11.6 




' Other games 


38 




34,2 


10.5 


34.2 


21.1 


Group Average 




41.5 


48.3 


39.1 


10.2 


40.8 


10.0 


Overall Average 




42.4 48.3 


38.9 


9.5 


41.4 


10.3 



ERIC 
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agreements consisted of "yes" re- 
sponses by clients coupled with 
"no" responses by attendants (41.1%/ 
on the average) , indicating power- 
ful acquiescence effects. More 
important is the observation that 
the two formats differ little; they 
are characterized by equally low 
agreement and equally strong re- 
sponse biases. 

Additional Compa.:ison 

A similar comparison of yes- 
no questions using pictures was 
conducted in the children's insti- 
tutional sample, where children 
were asked if they engaged in , ten 
different chores. Responsiveness 
averaged 69.8% for the ten pictures 
questions, vs- 70.9% for the words 
only format. Consistency of respond- 
ing between formats, averaged across 
the ten questions, was 87.9%- This 
consistency tended overwhelmingly 
to result from the client answering 
"yes" to both formats; 74.1% of all 
comparisons were consistencies of 
this type. For the picture format, 
client-attendant agreement averaged 
72,1%, resulting mostly from cases 
in which both client and attendant 
answered "yes." Most disagreements, 
'20.4% of all comparisons, occurred 
because the client responded "yes" 
and the attendant responded "no," 
indicating acquiescence. This pattern 
of responses was virtually repeated 
in the words only format, where over- 
all agreement was 69-5%. 

conclusions 

Yes-no questions using pictures 
tend tq^be invalidated by acquies- 
cence, as are similar questions using 
words only. The use of pictures does 
not increase responsiveness or in 
any way improve tne information ob- 
tained using yes-no questions. In 
short, the use of pictures offers 
no advantage over the simple words 
only format. 



The Limits of Acquiescence 

By now, the reader may be weary 
of our continual references to the 
problem of acquiescence. Yet it is 
of such importance that we feel a 
need to draw together some of our 
findings and present still more so 
that the implications of acquiescence 
are clearly appreciated. 

One of the pairs of oppositely worded 
questions that were used in all three 
major samples was "Are you usually 
happy?" versus "Are you usually sad?" 
Of those persons who could respond 
to both questions, fully 50.9% of 
the community children, 43.8% of 
the institutionalized children, and 
41.7% of the institution adults ac- 
quiesced, that is, answered "yes" 
to both questions. The correlations 
between acquiescence to this pair 
of questions and IQ in these samples 
were -.30, -.40, and -.41, respec- 
tively, all significant at the .05 
level and all suggesting that lower 
IQ respondents are more likely than 
higher IQ respondents to acquiesce. 
The oppositely worded questions 
"Are you usually by yourself?" and 
'"Are you usually with other people?" 
were also used with all three scimples. 
Here, 50.0% of the community children 
39.4% of the institution children, 
and^]11.7% of the institution adults 
contradicted themselves by saying 
"yes" to both questions. The correla- 
tions between acquiescence and IQ 
were -.24, .03, and -.04, respective- 
ly, with only the first statistically 
significant. Thus, in all three 
samples and on both pairs of ques- 
tions, acquiescence rates were alarm- 
ingly high. And although lower IQ 
respondents tended to acquiesce 
^ more, the correlations were not 
large enough or. stable enough to 
suggest that severely retarded per- 
sons are the only culprits. Instead'', 
acquiescence was a pronounced prob- 
lem of all levels of retardation. 
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still, having found some relationship 
between acquiescence and IQ, we 
explored the matter further in new 
interviews with 29 moderately and 
severely retarded institutionalized 
children and adults who had proven 
to be largely/ but not totally, unre- 
sponsive in their original inter- 
views. We included the "happy-sad" 



and "alone-with other" question? 
as well as other oppositely worded 
pairs, some of which were factual 
rather than subjective in nature. 
We/ also used a second approach to 
1 "smoking out" acquiescers, asking 
questions that demand a "no" answer. 
The results we obtained are pre- 
sented in Table 8.5/ ^ ^^^^ ^' 



Table 8.5A: Rates of Acquiescence in Response to Pai 



of Oppositely-Worded Questions 





Questions 


Responding 


Percent 
AcquicSL. iiiy 




1. 


Are you usually happy?" vs. Are you 
usually sad? 


23 


27.8% 




2. 


Are you usually with other people? 
•tre z^r**^ vou usually alone? 


25 


56.0% 




3. 


Most mornings here, do you pick 
out what clothes to wear? vs. 
Most mornings here, does somebody 
else tell you what clothes to wear? 




71.4% 




4. 


Do you livein G-10 (incorrect 
cottage number) right now? vs. 
Do you live in G- (client's 
cottage number) right now? 


25 


20.0% 




5. 


Here is a picture of (super- 
intendent' s name) . Is he the 
superintendent here at the 
colony? vb.. Here is a picture 
of (business manager's name). 
Is he the superintendent here 
at the colony? 


24 


83.3% 








Average 


: 55.7% 





r 
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they must be seen to be believed. 
Unless answers to yes-no questions 
were viewed with a jaundiced eye, 
one might conclude that a large 
proportion this sample consists 
of Chinese school bu6 drivers who 
fly airplanes on the side! It is 
important to note, however, that 
acquiescence rates varied from ques- 
tion to question. For example, al- 
though these reitpondents demonstrated 
an almost total lack of knowledge 
of the identity cf the superintend- 
ent of the institution and acquiesced 
in response to the pair of questions 
about this topic over 80% of the 
time, they apparently were more cer- 
tain of their cottage numbers and 
only 20% acquiesced when asked if 
they lived in another cottage. 



To exploro ti\e implications of these 
staggering rates of acquiescence, 
we examined the extent to which these 
same respondents claimed to' be in- 
volved in four chores in the cottage. 
The average respondent claimed to . 
be involved in 80% of the chores; . 
yet when we asked attendants the 
same questions, they claimed tiie 
average resident was involved in 
only 59% of the chores. More im- 
portantly, those respondents who 
showed themselves to be especially 
prone to acquiesce tended to claim 
more involvement in chores. If we 
had relied uncritically on these 
answers, we would have concluded, 
on the basis of a correlation of 
-.61, that lower IQ respondents 
were more involved iri chores than 



Table 8.5B: In Response to Questions for which 
"No" is the Correct Answer 



Question 


Responding 


% Yes Responses 


1. 


Are you Chinese? — 


25 


44.0% 


2. 


Does it ever snow here 
in the summer? 


26 


73.1% 


3. 


Do you know how to fly an airplane? 


25 


52.0% 


4. 


Are you a school bus driver? 


24 


41.7% 


5. 


Right now, is it raining outside? 
(or sunny outside, whichever question 
should be answered "no" at the time 
of the interview) 


25 


28.0% 






Average : 


47.8% 



higher IQ respondents. Yet this . 
finding would have been un arti- 
fact, for lower IQ respondents also- 
scored higher on our measures of 
tendency to acquiesce • Using attend- 
ant data in place of client data, 
the correlation between IQ and in- 
volvement in chores turned out to 
be .26 rather than -.61. Thus this 
. analysis not only confirmed the 
finding that acquiescence tends to 
be more likely among the more severely 
retarded but also suggested that 
their acquiescence can distort the 
relationship between IQ and the 
topics of interest in an interview 
(in this case, involvement in 
chores) . We trust that these addi- 
tional analyses leave no doubt 
about the magnitude of the problem of 
acquiescence. Responses to yes- 
no questions simply cannot be trusted. 

The Use of Pictures 
With Either-Or Questions 

' In a continuing effort to maxi- 
mize responsiveness while minimizing 
systematic bias, we tested some 
either-or questions which offered 
pictures as response alternatives, 
hoping to increase responsiveness 
to either-or questions without de- 
creasing their generally high 
validity. 

QUESTIONS ASKED OF COMMUNITY CHILDREN 

Illustrative Comparison 

Community children were asked 
whether they usually spend their 
time alone or with others, using 
the following set of four questions: 

When you're not at school^ are 
you usually by yourself or with 
other people? 

When you're not at school, are 
you usually with other people 
or by yourself? 
This picture shows a boy/girl 
who is by himself /herself and 



this one shows a boy/girl who 
is with other people. When 
you're not at school, which 
boy /girl is most like you? 
Point to the picture. 
This picture . . .(same>as 
above with order of response 
options reversed) 

Data collected from these four ques- 
tions are summarized in Table 8.6. 

As expected, the children were con- 
siderably more responsive to the 
two picture choice questions than to 
the verbal either-or questions. For 
the set of four questions, thire are 
six unique pairs, allowing for six 
consistency figures. One consistency 
comparison is a within-format com- 
parison between the two verbal either- 
or questions. The consistency figure 
for that comparison was only 58.1%. 
The largest fraction of inconsistencies 
for thi3 comparison (25.6% of response 
pairj?) occurred because the client 
chose the last option in both cases, 
suggestiivg a last option position 
preference. 

A second consistency comparison was 
between the two picture choice ques- 
tions. Consistency for this comparison 
was a respectable 72.7%. Inconsist- 
encies were nearly equally balanced, 
with a slightly larger' proportion 
of inconsistencies due to choice 
of the first option in both cases' 
(14.7% of comparisons), suggesting 
that response bias may be lessened 
or possibly reversed in th^ picture 
choice format. The high consistency 
of these questions casts picture • 
either-or questions in a favorable 
li^ht. 

OA 

The remaining four consistency com- 
parisons were between. picture choice 
and verbal eithe':-or formats. These ' 
consistency figures ranged from 
55.1% to 63.0%, averaging 59.3%.. 
The patterns of inconsistencies among 
these questions were not particularly 
revealing. 
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Table 8.6: Either-Or vs. Either-Or with Pictures 



Question 


Responsiveness 


Mean Consistency* 


% Agreement 
. -^^.^^ . 


When you're not at school, are you usually by 
yourself or with other people? 

1 


69.6% 


57.9% 


51.0% 


When you're not at school, are you usually 
with other people or by yourself? | 


80.0% ■ 


59.3% 


50.0% 


This picture shows a boy /girl who is by ^ 
himself /herself and this one shows ^ 
boy/girl who is with other people, tthen 
you're not at school, which boy/girll is 
most like you? Point to the picture. 

1 


94.7% 


65.4% 


• 41.5% 


This picture shows a boy/girl who is with \ 
other people, and this one shows a 
boy /girl who is by himself /herself . J 
When you're not at school, which j 
boy/girl is most like you? ! 

Pninl" t"n thp ni rtuT^. ' 1 

_ 1 

I ' 1 


94.7% 


62.2%' 

• 


50.0% 

\ ■ 
i 

■ 1 










1 

Average: 


84.8% 


61.2% 

1 


48.1% 




1 



I'JJ 



ill- 



ERXC'is ^iS"'"^ '5 ^^^^^9^ consistency comparisons witn the other three questions. 



Average parent-child agreement for 
the four questions was only 48. 1^, 
which was below the level of agree- 
ment expected if both parents and 
children selected their responses 
randomly. Further, the two formats 
differed little in terms of this 
estimate of validity. Agreement 
for the verbal either-or questions 
averaged 50.5%, as opposed to 45.8% 
for the picture choice questions. 
Thus, the major conclusion suggested 
by the agreement data is that neither 
format elicited substantial agree- 
ment with parents on this topic. 
Still, the findings point to the 
effectiveness of pictures as a sup- 
plement to these either-or questions, 
for they increased responsiveness 
and reduced a bias toward selecting 
the last option given. 

Additional Comparison 

A^ similar set of four questions, 
systematically varying both format 
and order of response options, was 
asked of community children. These 
questions asked if the child is 
usually happy or sad. Responsive- 
ness to the two picture either- 
or questions averaged 97.4%, ex- 
ceeding the responsiveness figure 
for the two verbal either-or^ ques- 
tions (90.0%). Consistency for the 
comparison of the two verbal either- 
or questions was 75.5%. Most in- 
consistencies (15.1% of compari- 
sons) involved choice of the first 
option in response to both questions, 
demonstrating that last-option bias 
for these questions is" not a very 
robust phenomenon. Within-format 
consistency for the two picture 
choice questions was 67.8%, with 
inconsistencies more or less evenly 
mixed. For the four comparisons 
of picture arid verbal either-or 
questions, consistency averaged 
72.3%, with no pattern readily ap- 
parent in the inconsistencies. Av- 
erage agreement between parents 
and children for the verbal questions 



was 79.0%, as opposed to 70.4% for 
the picture either-or questions. 

Conclusions 

Both comparisons indicate that 

. the use of pictures as response 
alternatives maximized responsive- 
ness to either-or questions, a 
finding consistent with nearly all 
data previously analyzed. As regards 
the questions of reliability and 
validity, no clear conclusions can 
be drawn from thes^comparisons,' 
which generated incon^54^tent re- 
sults. The first comparison suggested 
a reliability advantage for the pic- 
ture choice format, while the second 
comparison contradicted that result. 
Agreement data were also inconsistent. 
Agreement for the alone-with others 
comparison was so abysmally low that 
both either-or formats appear inade- 
quate. For the happy-sad comparison, 

V both formats appear in a favorable 
light, with verbal questions gaining 
a slight (79.0% vs. 70.4%) advantage. 
Overall- pictures clearly increase 
th^ ability of retarded persons to 
answer either-or questions. Since 
pictures do not appear to system- 
atically threaten validity, their 
use can be recommended. 

INTERVIEWS WITH SEVERELY 
RETARDED PERSONS 

Alternative format comparisons 
to , this point have left us -trapped 
by a dilemma; namely, the v^ery ques- 
tions that many severely riatarded 
persons can answer, particularly 
ye^-no questions, yield invalid 
answers, it appeared that picture 
choice either-or questions offered * 
a last hope of obtaining valid re- 
sponses from retarded persons with 
extremely limited verbal skills. 
For this reason, we made a final 
attempt to test the effectiveness 
of picture either-or questions as 
a means of gaining information from, 
severely retardeu individuals (chil- 
dren and adults in the institution 
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who had been relatively unresponsive 
when they were first interviewed) . 

Illustrative Comparison 

This comparison was identical 
to the happy-sad comparison conducted 
in the community children sample, ^ 
pitting a verbal either-or question 
and its oppositely worded counter- 
part against a picture choice §ither- 
or question and its complement. 

Responsiveness data for this com- 
parison are represented in Table ^8.7. 
Once again, the use of the pictures 



as response alternatives maximized 
responsiveness. Comparison of the 
two verbal either-or questions 
revealed consistency of 71.5%. 
Moreover, this comparison suggested 
that last option response bias 
played an important role, as all 
inconsistencies (28.6% of compari- 
sons) involved choice of the last 
option both times. Within-format 
consistency for the picture either- 
or questions was comparable (73.0%) 
but, importantly, gave no indica- 
tion of systematic bias. Of the 
seven respondents who gave incon- 



Table 8.7: Verbal and Picture Choice Either-Or Questions asked 
of Severely Retarded Persons 



Question 


Responsiveness 


% Agreement 




Are you usually happy or sad? 


62.1% 


"63.6% 




Are you usually sad or happy? 


75.9% 


96.0% 




Format Average: 


69.0% 


79.8% 




This picture shows a boy/girl who is 
happy, and this one shows a boy /girl 
who is sad. Which picture shows how 
you usually feel? 


82.8% 


73.1% 




This picture shows a boy/girl who is 
happy, and this one shows a boy /girl 
who is sad. Which picture shows how 
you usually feel? 


82.8% 

/ 


66.7% 




Format Average: 


82.8% 


'69.9% 




Overall Average: 


75.9% 


74.9% 
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sistent responses to the two ques- 
tions, four did so by selecting 
the first option both times / as 
opposed to three who chose the last 
option both times. Thus/ consist- 
ency data suggest that the two 
formats are comparable with regard 
to reliability, but that picture 
choice questions have an advantage 
in that they are not affected by 
strong response biases. 

Agreement data are also summarized 
ia Table 8.7. An apparent advantage 
exists for the verbal questions, 
as agreement for those questions 
averages 79.8% as compared with 
69.9% for the picture questions. 
However, agreement figures for the 
two verbal questions were extremely 
discrepant. Although attendants 
consistently responded '*happy," 
clients' responses changed as a 
function of the order of response 
alternatives. Thus-, a high agree- 
ment figure (96.0%) for the ques- 
tion "Are you usually sad or happy?" 
represents not response validity 
but an accidental correspondence 
of client response bias with attend- 
ants* response preference. This 
is evident because agreement for 
the oppositely worded questions was 
only 63.6% and. most disagreements 
involved choosing the last option 
("sad") while attendants said "happy." 
By comparison, agreement for the 
two picture choice questions was 
relatively consistent and not power- 
fully affected by systematic bias; 
thus it can be concluded that the 
picture choice questions generate 
reasonably valid responses and are 
less distorted by response bias than 
the verbal choice questions. 

Additional Comparisons 

Two additional comparisons eval- 
uated picture chbdce either-or ques- 
tions used in interviews with severely 
retarded persons. The first comparison 
was identical in form to the above 



coLiparison, pitting two pictorial 
against two verbal either-or questions, 
asking in each case whether the client 
is usually alone or with other people. 
As always, the advantage in eliciting 
appropriate responses accrued to 
the picture choice question, by a 
margin of 81.0% to 55.2%. Within- 
format consistency for the two ver- 
bal choice questions was 62.6%. Fully 
25.0% of comparisons were inconsis- 
tencies in which the client chose 
the last option both times, indicating 
the importance of last option response 
bias in this severely retarded group. 
The within-forraat comparison for 
the picture choice questions yielded 
a higher consistency figure of 88.0%, 
with no evidence of order effects. 
Agreement for these questions was 
quite low, averaging only 58,. 3% 
for the verbal questions and 53.8% 
for the picture choice questions. 
Analysis of the inconsistencies re- 
vealed that they were in each case 
almost perfectly split . between first 
and last choice responses by clients. 
On both verbal and pictorial questions, 
clients were much more inclined 
than attendants to respond "alone," 
implying that this is a genuine 
difference of opinion between clients 
and attendants which is not a matter 
of response set. Whether this dis- 
agreement involves invalid responding 
by clients, a lack of accurate in- 
formation on the part of attendants, 
or is simply a real disagreement 
is impossible to determine. 

The second comparison compaied two 
picture choice either-or questions 
with three yes-no questions asking 
whether the client or someone else 
usually picks the client's clothes. 
The pictorial either-or clothes 
questions yielded an average re- 
sponsiveness figure of 82.8%, which 
exceeded average responsiveness 
for the yes-no questions (57.8%). 
The comparison between the two 
picture-choice questions yielded 
a consistency figure of 62.5%. In 
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thi<= case, 29.2% of the response 
pairings involved choice of the 
first' option both times / suggesting 
that a first. option response bias 
may sometimes be involved in picture 
choice questions. Analysis of agree- 
ment yielded results similar to 
those from the above comparison; 
Agreement was only 52.9%, and in 
both questions clients were- much 
more likely than attendants to assert 
that they chose their own clothes. 

Conclusions 

The one unequivocal conclusion 
which can be drawn from comparisons 
of verbal and picture choice either- 
or questions is that picture choice 
questions elicit appropriate re- 
sponses from a larger fraction of 
the population. Many of these com- 
parisons also suggest impressive 
reliability of responses to either- \ 
or questions, and particularly to 
picture choice either-or questions, 
which are sometimes less affected 
by response' bias than are their ver- 
bal counterparts. These data indicate 
that the use of pictures does not 
reduce validity, although in some 
cases it introduces a first option 
response bias which parallels the 
last option bias found on verbal 
either-or questions. Overall, picture 
choice questions appear to hold promise 
greater than that of any other tech- 
nique we have used as a method of 
gaining information from more severely 
retarded persons. 

Conclusions 

This chapter has examined the 
advantages and disadvantages of al- 
ternative approaches to obtaining 
information from mentally retarded 
persons. Viewed in conjunction with 
data presented in earlier chapters, 
it enables us to formulate a set 
of empirically-based recommendations 
to follow if one wants to obtain 
valid answers from mentally retarded 
interviewees. 



Broadly, our findings establish that 
what one hears from mentally retarded 
piersons depends in part on how ques- 
tions are asked. We have demonstrated 
that not only question format but 
question wording and whether or not 
one uses probes after a question 
is asked can have dramatic effects 
on the answers that are given. As 
a result, the central recommendation 
of this study is as follows: Whenever 
mentally retarded persons are to 
be interviewed, there must be careful 
attention to methodological issues, 
particularly question design. The 
reader who has followed our narrative 
this far can scarcely be Surprised 
and may even consider the broad re- 
commendation so basic that it does 
not need stating. Yet, despite the 
fact that many studies relying on 
interviews with retarded persons 
have been conducted, this is the 
fitst time, t,o our knowledge, that 
this crucially, important maxim has 
been stated. It is certainly the 
first time that mieasurement issues 
in interviewing retarded persons 
have been systematically studied. 
It is now time to end the tradition 
in the mental retardation field 
of simply assiwing that responses 
to interviews are valid. 

Based on the findings we have pre- 
sented, we are able to offer the 
following more specific generaliza- 
tions and recommendations. 

1. Question wording matters. For 
example, if a question calling for 
an enumeration of activities in 
a given category (e.g., sports) 
mentions examples (e.g. ,. football 
and baseball) , retarded persons 
will be more likely than usual to 
name the very activities used as 
examples. Thus this strategy for 
making a question more concrete 
is counterproductive and should 
not be used. In addition, presum- 
ably small changes in question 
wording can have major effects on 
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response patterns. For instance, 
*we fo\ind that retarded persons were 
far more likely to say certain un- 
acceptable behaviors (e.g., hitting 
others) were prohibited if the ques- 
tion was phrased, "Are you allowed 
to. . than if the question was 
phrased Vis it against the rules 
to. . ."We even found, thanks to 
a typographical error in the inter- 
view schedule, that responses to 
"Who decides what chores you, do?" 
and "who decides what chores to 
do?" differed. In addition, we found 
evidence that retarded persons do 
not always pick up distinctions 
in wording that the researchers 
hope they will; for example, dis- 
tinctions among quantitative words 
such as "always," "usually," "some- 
times," and "never." Finally, we' 
found abundant evidence that responses 
to oppositely worded questions (e.g., 
"Are you usually with other people?" 
vs. "Are you usually alone?") can 
differ drastically. There are many 
examples that we can cite of mis- 
interpretations, often humorous ones, - 
of words and phrases in questions 
(see Sample 8.4) As we are not cer- 
tain of all the possible effects 
of question wording, we can only 
recommend that there be extensive 
piloting of interview schedules to 
determine whether respondents under- 
stand and, if so, how they interpret ' 
alternatively phrased questions. 

2. ^Techniques of probing for addition- 
al information must be carefully 
planned. As we have shown here, when 
interviewees are asked open-ended 
questions calling for an envmieration 
of activities, many cannot respond 
at all and those who can are likely 
to name very few activities. On the 
other hand, the strategy of probing 
indefinitely with "What else?" solves 
that problem but creates another 
that is more serious: ovd^reporting 
of activities in response to the 
implied demand to do so to the ex- 
tent that answers lose validity. 



Thus, if such probes are used, and 
perhaps they should be to overcome 
the problem of underreporting on 
open-ended questions, they should 
probably be used only once or twice , 
not ad infinitiam. Moreover, probes 
or follow-up questions should be 
phrased to hark back to the content: 
of the original question (e.g., in- 
stead of "What else?" "What else 
r do you do at your job?"). This strat- 
egy of reminding interviewees of 
the question content increases 
the ability of inter /iewees to answer, 
without reducing validity--of\ response. 

More generally, all 'of our findings on 
question word^ngTand format need to be 
applied'-wh'eh probes or follow-up 
questions are used. Probes and follow- 
up [.questions should, as a general 
rule, be as carefully planned as 
the original questions they follow. 
J^ea^7^ng the design of such questions 
to "interviewers who are insensitive 
to the implications of question 
structure and wording is essentially 
leaving matters to chance. If one 
interviewer chooses to follow up 
an unclear answer with a simple 
yes-no questions while another chooses 
to use an either-or question to, 
pursue the matter, we can predict 
that the two interviewers are likely 
to get different ansv/ers. This .is 
by no means a trivial issue, for 
our experience suggests that no 
matter how well original questions 
are designed, there J^rikely to 
be some need to' follow-up or clarify 
vague or confusing answers.- 

3. Multiple choice questions that 
call for responses along a quanti- 
tative dimension (e.g., a lot, some, 
not much, and not at all) appear 
to be generally useless as sources 
of information. They are relative- 
ly difficult for retarded persons 
to answer, in the first place; but 
more importantly, respondents appear 
to have difficulty attaching con- 
sistent meanings to' such teinns. 
We did not find any advantage in 
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Sample 8.4 



A Sctmpling of Unusual Interpretations of Questions 

1. Q: Which picture shows how you like the food here? 

Point to the picture. 
A: I know, keep your mouth closed. 

2. Q: Do you decide what chores you do? . 

A: Probably clean house and fix your bed and clean up hallway 
and mop and go walking and go see somebody's house- 

3. Q: Are you allowed to hit people? 
A: I don't hit people. 

4. Q: What do you and your friends usually do together? 

A: Let's see. My mom says.. . (Puts hand to head aS if in 
deep thought). Let me think. Well, I say yes. 

5. Q: Do people here help you when you want help? 
A: No, I help myself to bed, go to the bathroom. 

6. 0: If you had one wish, what would you wish for? 

A: Well, I go to school and we're going to have a party this, 
afternoon. 

7. Q: Do you decide what chores you do? 

A: We go to the canteen and get something. 

8. Q: Do you decide what chores you do? 
A: Not until I'm assigned to it. 

9. Q: Do people here taice things c.way from you? 
A: No, but they do steal. 



10. Q: Do you dust furniture? 

A: No, wash ears, that*s all. 

11. Q: Are yoii allowed to go on dates here? 
A: No, my daddy is. 

12. Q: Has anybody from Voc. Rehab, ever tried to help you? 
A: (nodded yes) 

Q: How did they help you? 

A: Wash your head. 

13. Q: Where do you get your money? 
A: My money, in my purse. 

14. Q: Do people here make you do things you don't want to do? 
A: It's not wise. 

15. Q: Could you buy some candy with $1.00 (question designed to 

check money 'concepts) 

A: I done it, but I had pimples once before. 

Q: (repeated) 

A: If I don't get a hot dog. 

16. Q: Do you usually have problems knowing what to do when you 

get sick? 

A: No, always run to the bathroom and get sick. 
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breaking up a four-choice multiple 
choice question into a yes-no ques- 
tion and then a multiple choice 
question (e.g. , "Do you Tc^ave here 
to go see your famu.l'y?" if yes "Do 
you go see them a lot, sometimes, 
or not much?"). Nor did we find 
smiling and sad faces as an alter- 
native to verbal multiple questions 
about satisfaction useful, except 
to" boost responsiveness. / 

unfortunately, we have no con- 
structive advice for interviewers 
who seek quantitative information 
about such things as extent of 
involvement in various activities. 
Based on previous experience in 
other research projects with ques- 
tions calling for estimated numbers 
of times doing something in a given 
time period, we steered away from 
that approach. We simply do not feel 
that retarded persbns typically have 
a good enough grasp of quantitative 
concepts and time concepts that they 
can respond validly. 

4. Multiple choice questions offer- 
ing discrete response alternatives , 
however, may have promise. In asking 
respondents what type of dwelling 
they lived in and how they got to 
school, we found that questions of 
this type worked well; better than 
open-ended questions on the same 
topics, for they enhanced responsive- 
ness without sacrificing validity. 
Although responses to verbal multi- 
ple choice questions were sometimes 
more affected by the order of options 
than were responses to parallel pic- 
ture choice questions, we are not 
confident that pictures p3:ovide 
enough of an advantage to warrant 
th° effort involved in producing 
them. Moreover, we are not able to 
say whether multiple choice questions 
naming discrete response options would 
work in other content areas, as we 
used them only to get information 
about basic, and presumably well- 
known, factual inforroation. 



5. Yes-no questions, despite the fact 
that they are easily answered by most 
retarded persons, are not useful in • 
interviewing retarded persons. While 
this statement may seem strong, we 
have devoted more attention to yes-no 
questions than to any other format 
and we have found, with few exceptions, 
that retarded persons are highly like- 
ly to answer "yes" regardless of ques- 
tion content. Some of our evidence of 
acquiescence in response to yes-no 
questions cannot be described as any- 
thing other than shocking. In compari- 
sons of oppositely worded yes-no ques- 
tions, it was not uncommon for the most 
frequent pairing of responses to the 
two questions to be "yes" to both 
rather than the expected "yes" to 
one and ''no" to the other. We have 
witnessed severely retarded persons, 
asked questions that should be an- 
swered "no," claim through their 
yeasaying, to be Chinese school 
bus drivers who fly planes on the 
side. The impact of acquiescence 
is so strong that we feel yes-no 
questions should be avoided entirely. 
If they are used, we can see no 
alternative but to build into the 
interview some check on acquies- 
cence (see Chapter 9 for some sug- 
gested strategies) . 

6. Open-ended questions have one 
overriding limitation: they cannot 
be answered by very many retarded 
persons. As a result, they are not 
very useful at all in interviewing 
more severely retarded persons. 
However, they can be of use in inter- 
viewing persons with relatively 
advanced verbal skills. We found 
that simple open-ended questions 
in which a single response can be 
coded into categories (e.g., "Who 
decides what chores you do?") yielded 
relatively valid answers from those 
who could respond. Open-ended ques- 
tions calling for enumerations (e.g., 
"What things do people here do that 
bother you?") were less satisfactory. 
In comparisons of them with parallel 
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sets of yes-no questions, we found 
that they probably led to underre- 
porting, although it was not so 
pronounced as the overreporting 
in response to yes-no questions. 
Very few retarded per sons _apBear • 
to have the level of verbal skill 
that such enumeration requires. 
There is often no choice but to 
ask some open-ended questions in 
interviews. As long as the respondents 
are verbally skilled, these questions 
can yield great insight into the 
lives of retarded persons. The only 
caveat ig that the insights gained 
will probably not be representative 
of the larger population of persons 
one is studying. Thus we can best 
envision open-ended questions being 
used as fdl low-up questions to more 
Structured questions, to gain addi- 
tional clarification of answers to 
those structured questions from those 
capable of providing it. 

7. Of all the question formats we 
have tested, either-or questions, 
particularly those accompanied by 
pictorial representations of the 
alternatives, appear to be the best 
way of optimizing responsiveness 
and response validity. In numerous 
•comparisons of either-or and yes- 
no questions, either-or questions 
consistently emerged victorious. 
While they are somewhat more diffi- 
cult to answer than yes-no questions, 
this slight sacrifice in responsive- 
ness is more than made up for by 
a gain in response validity. We have 
found some evidence of a preference 
for the last of the two options pre- 
sented, but it has not been totally 
consistent and, more importantly, 
it has typically been a minor re- 
sponse bias in comparison to acquies- 
cence. Furthermore, the use of pic- 
tures as adjuncts to either-or ques- 
tions appears to be advantageous. 
It boosts responsiveness to the point ■ 
that all or virtually all persons 
can respond and results in no major 
sacrifice of validity. If anything, 
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the use of pictures reduces the ten- 
dency to select che last option in 
either-or questions, this appeared 
to be particularly true among severe- 
ly retarded persons, among whom re- 
sponse biases are generally more 
pronounced. Clearly either-or ques- 
tions are not a panacea, for there 
are many t-opics that simply do not 
lend themselves to the either-or 
format. However, they should be given 
a great deal of consilderation as 
viable alternatives to yes-no ques- 
tions. For example, in\ a severely 
retarded group one could count on 
getting a huge number of "yes" an- 
swers to the questions, \ "Do you cook 
on a stove?" It would be^ possible X 
to structure a pictorial ^either- / 
or question on the same tropic that, 
v/ould in all likelil^ood veld ih- 
fomation that is much closer /to 
the truth; for example, "H^r/ is 
a boy who cooks on the stoj>e. Here 
is a boy who does not copk on the 
stove. Which boy is li^ke you? The 
one who cooks onujth^^stove or the 
one who doesi^t cook on the stove? 
Point to the picture." 

In Chapter 9 we will spell out more 
of the implications of our findings 
for persons who solicit informa- 
tion from retarded persons by ques- 
tioning them. As the list presented 
here suggests, our study generally 
revealed more about how not to ask 
questions of retarded persons than 
how tc do so most effectively. Still, 
we believe that knowing what to 
avoid makes progress more likely. 
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Chapter 9 



CONCLUSIONS AND IMPLICATIONS 



/ 



This report has covered k great 
deal of ground in detailing /the find- 
ings of a complex project that was 
designed to explore the feasibility 
of interviev/ing as a means of obtain- 
ing information directly from mental- 
ly retarded persons. In an attempt 
to bring closure to the report, we 
will briefly review the study and its 
major findings, discuss recommenda- 
tions for conducting survey research 
with mentally retarded persons, and 
point out some general implications 
for researchers and practitioners in 
the mental retardation field. 

s\immary of the Sjtudy 

WHAT DID WE DO? 

\we set out to answer a question 
originally posed by the President's 
CommiVtee on Mental Retardation: Is 
a national polling of a sample of the 
mentally retarded population a fea- 
sible m^ans of gathering consumer 



input? We approached this guiding 
question\by addressing the following 
more specVfic questions: 



1. To what extent can retarded 
persons respond to questions 
in an appropriate fashion, and 
what factors affect their 

re spon s i vene s s ? 

2. How reliable are their re- 
sponses, in the sense of being 
consistent over short periods 
of time? 

3. How valid are their responses, 
in the sense of being free of 
systematic bias and agreeing 
with information provided by 
parents or caretakers or docu- 
mented in records? 

4. What types of questions appear 
to optimize responsiveness, 
relicibility , and validity? 

We began with reviews of the litera- 
ture on the language and communica- 
tion skills of retarded persons and 
response biases in survey research 
with the general population. We also 
explored issues in sampling and gain- 
ing access to retarded persons for 
purposes of interviewing them. Out 
literature reviews suggested that. 
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for the most part, retarded persons 
are likely to behave linguistically 
like children at various stages in 
the development of mature language 
competence, -^nd that, moreover, many 
retarded persons have hearing and 
speech handicaps that may pose prob- 
lems in an interview. In addition, 
we found cause in the literature on 
response biases in the general popu- 
lation to expect retarded interview- 
ees, like children and less educated 
adults, to be especially prone to 
give biased answers; for example, to \ 
acquiesce in response to yes-no 

questions. I 

\. 

Our irajor mission then bejdame one o^ 
systematically testing inite^viewing 
approaches with diverse samples of 
mentally retarded children and adults. 
Interviev; schedules focused on the 
topics of living circumstances, daily 
activities, utilization of services, 
and involvement in decision -making. 
Five samples of retarded persons were 
interviewed. The three primary sam-^" 
pies consisted of 52 institutional- 
ized children 12-16 years old, 58 
institutionalized adults, and 57 
children living in the comi^unity. 
Further reseaivch was done with 29 
institutionalized and primarily 
severely retarded children and adults 
who had proven largely but not total- 
ly -unresponsive in the first inter- 
vie^vs, and -a small sample of 13 
adults living in the community and 
enrolled in a citizen advocacy pro- 
gram. 

Each study also involved parallel 
interviews with "significant others": 
parents, attendants, or citizen ad- 
vocates, depending on the sample. 
These persons were asked to answer 
questions paralleling most of the 
questions that had been asked of the 
retarded respondents so that the 
responses of retarded persons and 
nonretarded informants speaking for 
them could be compared. Finally, 
botfi institutionalized samples were 
interviewed twice, a week apart, so 



that the reliability of their re- 
sponses to the same questions asked 
cn two different occasions could.be 
examined. In all,' well over 500 
interviews were conducted. Alterna- 
tive questions about the same topics 
were embedded within interview 
schedules to aid in identifying 
question formats and phrasirigs that 
optimize responsiveness, reliability, 
and validity of response. Much of 
the analysis focused on responses to 
individual questions, comparing an- 
swers given at different times, an- 
swers given to alternative questions 
on the same topics, and answers given 
'by retarded persons and parallel 
answers given by parents or attend- 
ants. In addition, some .composite 
scores were formed and used in 
correlational analyses sO that in- 
dividual differences in interview, 
performance could be examined. 

TO highlight so^e of the major find- 
ings, we will offer concise answers 
to the four major questions of the. 
study . j 

TO WHAT EXTENT CAN RETARDED PERSONS 
RESPOND TO INTERVIEW QUESTIONS? 

In general, we found that many 
retarded persons are capable of 
giving appropriate answers to simply 
worded questions. The measure of 
responsiveness to questions that we 
typically used was the percentage of 
occasions on which a response met, 
at least minimally, the • formal de- 
mands of the question (e.g., a "yes" 
or "no" to a yes-|p^question or 
mention of one activity in response 
to an open-ended question calling 
for an enumeration of activities) . 
We obtained at least minimally appro- 
priate responses at least two-thirds 
of the time in our samples of severe- 
ly to mildly retarded persons. Of 
the various inappropriate responses 
^e encountered, the- most frequent 
appeared to be failure to give any 
response; vague answers that could 
not be coded into content categories 
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(e.g., equivocation on yes-no' ques- 
tions to the extent that the response 
could not be classified as "yes" or 
"no"), unintelligible responses^ and 
irrelevant responses. An individ- 
ual's ability to respond appro- 
priately to questions was found to be 
highly stable from week to week, as 
indicated by correlations of .96 
between percentages of answers that 
were appropriate in two interviews 
held a week apart with insuitutional- 
ized dhildren and adults. 

Mo^t importantly, we found it pos- 
sible to predict how responsive 
individuals would be. First, respon- 
siveness appears to be a function of 
IQ, ^ As we had expected, it is all 
but impossible to interview most 
profoundly retarded persons. In our 
one attempt to interview, profoundly 
retarded persons, only three of the 
16 adults approached passed a simple 
screening interview and supplied any 
answers at all to the full ifiterviews. 
Sever'^ly retarded persons were di- 
verse in their ability to respond tc 
questions, and the extent of their 
limitations comparcid to moderately 
and mildly retardev- persons varied 
somewhat from sample to sample. 
Generally, however, responsiveness 
was greatest in the moderately and 
mildly retarded ranges. Overall, 
the correlations between IQ and re- 
sponsiveness ^scores in the three 
major samples varied from .35 to .67. 

second, the ability to respond is a 
function of the type of question 
asked. Consistently in our samples, 
high numbers of interviewees were 
able to respond appropriately to 
simple yes-no questions and to ques- 
tions involving choices among pic- 
tures in which a nonverbal response 

^(pointing) would suffice. Either-or 
questions were the next easiest to 
answer, and open-ended questions and 

'multiple choice questions with three 
or four structured options were the 
most difficult for interviewees to 
answer. 



Finally r we found tnit responsive- 
ness could be increased by simply 
repeating a question if the first 
Xesppnse was less rnan minimally 
appropriate. In fact, some persons 
became more responsive to questions 
after having been interviewed with a 
similar instrument a week before. 
All of this suggests that ^fetarded 
persons may need additional training 
and experience in being interviewed 
in order to learn hew to respond 
appropriately, but that even during 
an interviev; they may benefit from 
additional chances to comprehend the 
question through having it repeated. 

However, one of the major assump-* 
tions of the study was that obtaxn- 
ina answers is not enough. One must 
also determine whether answers given 
are reliable and valid. 

HOW RELIABLE ARK ANSWERS? 

In general, we can say that the 
answer given one week by a mentally 
retarded person will typically 
correspond to the ansvrer ro the same 
question given a week later. In 
both institutionalized samples, an- 
swers given a week apart were gen- 
erally consistent ever 80% of the 
time. This was certainly true of 
most yes-no, either-or, and open- 
ended questions. However, we found 
reliability very low for picture- 
choice questions about satisfaction 
with institution life and for multi- 
pleXchoice questions involving 
quan\:itative terms such as "a lot" 
and "sometimes." Finally, although 
we had expected answers to subjective 
questions to be relatively changes- 
able, reliability figures for sub- 
jective yes-no questions were no 
lower than those for factual yes-no 
questions. 

Overall, then, one can generally ex- 
pect the picture of a group obtained 
one week to* correspond to the picture 
of that group obtained the next week. 
One can expect retarded individuals to 
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"stick to their guns'* la answering most 
of the time. We were\not able to iden- 
ify individual^ characteristics such as 
10 that are closely related to the ex- 
tent to which an individual responds 
reliably. , Moreover, we encounjtered 
many problems in interpreting reliabil- 
ity figures. And finally, as any re- 
searcher knows, reliability is a 
necessary precondition of validity 
but is not sufficient to establish 
validity. Thus, we had to attempt to 
determine whether answers given were 
not only reliable but also valid and 
meaningful. 

HOW VALID ARE ANSWERS? 

The issue of response validity 
was addressed through two approaches. 
First, we compared responses of re- 
tarded persons to independent sources 
of validating information. Typically, 
this took the form of comparison? 
with the responses of attendants or 
parents to parallel questions, al- 
though some answers could also be 
compared to documented fact. Overall, 
retarded interviewees and attendants 
or parents agreed (gave matchirig 
responses) about two-thirds of- the 
time in our various samples/^ At 
times, despite disagreement between 
individual pairs of clients and in- 
formants, pictures of the sample 
obtained from the two sources did not 
differ greatly. In such cases, one 
could at least use client and infor- 
mant data interchangeably to de- 
scribe a sample or to do analyses of 
individual differences. At other 
times, as a consequence of disagree- 
ment between pairs of clients and 
informants, pictures of the group 
obtained 'from the two sovtrc^s were 
widely discrepant. As the ov^all 
.finding of agreement about two- 
thirds of the time implies, it cannot- 
be safely assumed that interviews^ 
with retarded persons will yield the 
same information as interviews with 
their caretakers or parents. This 
is true even when there is no reason 



to suspect that clients' responses 
are systematically biased. It is 
particularly true when there is 
reason to believe that response 
biases such as acquiescence are at 
work . 

Agreement with parents or attendants 
proved to be an imperfect index of 
response validity for two reasons. 
First, the two groups simply had 
different perspectives oti some issues, 
and we were unwilling to infer that 
the retarded respondents were "wrong." 
Second, even high agreement was not 
alvays conclusive evidence for the 
validity of the responses of mentally 
retarded persons, for their answers 
sometimes revealed the workings of 
systematic bias. Thus, we took 
systematic biases in the response 
patterns of mentally retarded respond- 
ents into account in assesssing re- | 
spon*se validity. We did so princi- 
pally by comparing the responses of 
retarded \persons to two or more 
questions on the same topic but with 
different wordings or formats. Where 
changing the way the questions were 
asked powerfully altered the content 
of responses, we inferred that 
systematic biases were reducing 
validity. Systematic rather than 
random discrepancies between. the 
responses of retarded respondents 
and parents or attendants were inter- 
preted in the same way. For example, 
we typically found clients far more 
likely to answer "yes" on yes-no 
questions than informants were, and 
this pattern indicated that ac- 
quiescence on the part of retarded 
persons represented a threat to va- 
lidity of response. 

In short, we combined agreement 
figures and evidence for systematic 
response biases in assessing response 
validity. In the case of yes-no 
questions, in spite of relatively 
high agreement figures, clear evi- 
dejnce of acqui^sc^G€uJ^d us to con- 
clude that the responses"o€..^clients 
are largely invalid. Even wheif^"-^^^^^ 
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. agreement was high, we ty.pically 
found more clients saying "yes" than 
attendants or parents. Moreover, 
in response to pairs of yes-no ques- 
tions with reversed wording (e.g., 
"Are you usually happy?" vs. "Are you 
usually sad?") , retarded respondents 
frequen* ly contradicted themselves 
by responding "yes" to both questions 
of the pair. Quite importantly, 
correlational analyses demonstrated 
that agreement percentages for ques- 
tions increased as the percentage of 
parents or attendants saying "yes" 
increased; cl'ear indication that 
agreement figures were often artifi- 
cially high simply because acquies- 
cence on the part of clients happened 
to put them in agreement with infor-' 
mants. 

Open-ended questions provide another 
example of a question format for 
which high agreement was not 
necessarily good evidence for re- 
sponse validity. The vast majority 
of agreements involved both client 
and informant failing to mention 
something in a category of response. 
Somewhat more encouraging results 
were obtained with simpler open- 
ended questions that called for one 
answer rather than an enumeration of 
responses. Multiple choice ques- 
tions with quantitative response 
alternatives (a lot, some, not much, 
etc.), as well as multiple choice 
c[uestions calling for a selection of 
happy or sad faces to indicate a 
degree of satisfaction, appeared to 
have very Mttle validity at all- 
Finally, either-or questions, as we'll 
as factual multiple choice questiohs 
with discrete rather than quantita- 
tive options, appeared to have rela- 
tively good validity. Despite some 
evidence of a tendency to prefer the 
last option in such questions, there 
typically was not a marked bias in 
-the inconsistencies that occurred. 
Thus, we emerged from this analysis 
most confident about the validity of 
responses to open-ended questions 



calling for a single answer, either- 
or questions, and multiple choice 
questions with discrete response 
alternatives. 

We should note that analysis of re- ■ 
sponses to questions whose answers 
were documented fact did little to 
increase our confidence in answers 
given by retarded persons. Discour- 
agingly high proportions of respon- 
dents were not able to supply fully 
correct answers to questions about 
their full names, birthdates, 
addresses,, and so on. Many others 
could not respond at all to such 
questions. Analysis of these ques- 
tions bolstered the conclusion that 
the validity of information provided 
by retarded persons can never be 
taken for granted. Moreover, it 
suggests the practical need for 
training retarded persons so that, 
particularly in emergencies, they 
can ^communicate basic facts about 
themselves accurately. 

Finally, we were generally unable, to 
predict which persons would tend to 
give valid answers and which would 
not. Surprisingly, children living 
in the community were no ~mpre likely 
to agree with their parents (at least 
on a ^set of yes-no questions) than 
institutionalized persons were to 
agree with their attendants. Persons 
with higher IQs were more likely to 
agree with their parents or attend- 
ants than persons with lower IQs, 
but the correlation was not strong 
enough that we could confidently 
predict whose answers can be trusted 
and whose cannot. Until further 
research is conducted, one must be 
wary of responses given by any re- 
tarded per sort/ especially on the 
types of questions that are most 
likely to elicit systematic response 
biases. 

WHICH QUESTIONING APPROACHES " 
ARE MOST USEFUL? 

A major undertaking in this proj- 
ect was to conduct head-to-head 

.5 



205 



competitions between alternative 
question formats and phrasings in 
order to determine, with the topic of 
questioning controlled, which alter- 
native approach optimized responsive- 
ness and response validity. /To 
examine response validity, we again 
compared answers^ given by retarded 
persons and either attendants or 
parents, and we also looked for signs 
of systematic bias when the answers 
of clients to alternative questions 
were compared. ^ Since Chapter 8 
included a rather detailed summary 
of the findings of these analyses, we 
will only briefly review the recom- 
mendations that were made. 

1. Careful attention should be 
devoted to question wording. 
Seemingly small changes in 
wording can have large effects 
on responses. For instance, 
the use of examples of the 
topic of inquiry in the ques- 
tion (e.g., mention of football 
and baseball as examples of 
sports to be enumerated) 
biases responses such that the 
examples are mentioned more 
often than they would other- 
wise be. 

2. In probing for additional in- 
formation on open-ended ques- 
tions by asking "What else?" 
the probing should not be 
carried on indefinitely until 
the respondent can think of no 
more, for this strategy appears 
to threaten validity of re- 
sponse. It is probably wiser 
to use such probes only once 

^ or twice. * Moreover, when 
probes ate used, they should 
restate the content. of the 
original question to increase 
understanding (e.g. , "What 
else do you usually do for 
fun?" rather than "What else?"). 

3. Multiple choice questions with 
quantitative response options 
which are designed to get at 



extents of involvement in 
various activities appear to be 
generally useless, for retarded 
persons appear to have diffi- 
culty attaching consistent 
meanings to such terms as "a 
lot , " and "not much. " Simi- 
larly, pictures of happy and 
sad faces do not aid in elicit- 
ing information about satis- 
faction with life. However, 
multiple choice questions about 
basic facts that offer discrete 
response alternatives (e.g., 
walking, riding the bus, riding 
a bike, and riding in a car as 
ways of getting to school) do 
appear to be useful. 

4. Yes-no questions should prob- 
ably be avoided in interview- 
ing retarded persons because 

of the often disastrous effects 
of acquiescence on response 
validity. Pictures as accom- 
paniments to such questions 

apparently do nothing to. 
prevent acquiescence. 

5. Open-ended questions have the 
major disadvantage of not being 
answerable by high proportions 
of mentally retarded persons. 
Our tests of questions calling 
for enumerations of activities 
suggested that these questions 
yield very little information 
from most people and probably 
result in underreporting of the 
activities inquired about. By 
contra^st, open-ended questions 
calling ^or single answers 
generally y^ield valid responses^ 
assuming they can be answered, 
and appear to\be appropriate in 



interviewing 
persons. 



Lgher IQ retarded 



6. Either-or questions emerged as 
the single most satisfactory 
type of questioning strategy. 
Although they aire not quite as 
easy to answer as yes-no ques- 
tions (indeed, /several respon- 
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dents answered them with a 
"yes") , responsiveness to them 
was still relatively good, and 
more importantly, clients' 
answers and informants* answers 
tended to be in agreement. The 
only negative finding was a 
tendency , but not a strong or 
totally consistent one, toward 
selection of the' last of the 
two options presented, a re- 
sponse bias that would need, to 
be guarded against, Finr.lly, 
it was in conjunction with 
either-or questions that pic- 
tures appeared to have their 
greatest effect. The use of 
pictures with either-or ques- 
tions consistently increased 
responsiveness to the questions 
without sacrificing validity 
(and in a few cases enhancing 

it by counteracting the bias 
in favor of the last response 
option) . 

We are painfully aware of the fact 
that our research has typically told 
us a great deal more about how not 
to interview retarded persons than 
how to do so effectively. As the 
list above suggests, we continually 
encountered threats to response 
validity that forced us to raise 
grave questions about the utility of 
some questioning tactics. Even for 
the best of question formats, validity 
was often marginal. Nonetheless, the 
findings do offer guidelines for con- 
structing interview schedules ;to 
optimize the chances of getting an- 
swers, and more importantly, getting 
answers that are meaningful reflec- 
tions of the attitudes, needs, and 
circumstances of retarded citizens. 

DIRECTIONS FOR FUTURE RESEARCH 

The present study breaks new 
ground by exploring in a systematic 
way methodological issues in inter- 
viewing retarded persons. Obviously 
the answers we have been able to 
provide are less than complete; more- 
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over, our study has raised at least 
as many questions as it has answered. 
It has at least established unam- 
biguously the fact that the validity 
of answers given by retarded persons 
can never be taken for granted. 
However, many of the specific factors 
which influence the responses of 
mentally retarded persons remain un- 
explored- We would encourage other 
researchers in the mental retardation 
field to continue this promising line 
of research in hopes of enabling both 
researchers and practitioners to 
obtain useful information from re- 
tarded persons. We reconmiend careful 
investigation of the entire range of 
factors that may affect responses. 

Our study has focused almost exclu- 
sively on characteristics of inter- 
views that affect responses, primari- 
ly question format and ^phrasing. 
Other issues in. interview design 
remain to be addressed. For example, 
our study suggested that social 
desirability may enter into the 
responses of retarded persons to many 
questions, but we did not systemati- 
cally deal with that issue. ^ Do 
mentally retarded persons have a 
heightened motivation to give social- 
ly approved answers, and are their 
responses especially biased on items 
that raise issues of self -presenta- 
tion? A careful investigation of 
question difficulty, perhaps empha- 
sizing vocabulary difficulty and 
conceptual complexity, might also 
prove fruitful, as this was only in- 
cidentally examined in /the present 
study. \ / 

Moreover, as Sudman kr{d Bradburn 
(1974) indicate, respondent variables 
arid interviewer varia^-bles, along with 
task variables such ks question 
design, can introduqe error into 
survey research. We have explored 
the respondent variable of IQ in 
some detail, and have given some 
attention to other' respondent vari- 
ables such as age, sex, and institu- 
tionalization. Although we found 
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these additional respondent variables 
to be of minor importance, more 
intensive investigation of them might 
be warranted. The fact that we have 
found lower IQ individuals less 
responsive, more likely to acquiesce, 
and somewhat less likely to agree 
with par^nji^s and attendants empha- 
sized the potei^tial importance of 
respondent variables. As for inter- 
viewer variables, we left the issue 
entirely unexplored. For example, 
do interviewers wl;io are already 
familiar to retarded interviewees 
ob^in different information than do 
outsiders? How do the interviewer's 
attitudes and behaviors affect 
responsiveness and the nature of 
responses? The literature on threats 
to response validity in survey re- 
search with the general population 
(See Chapter 2) provides a ready 
source of leads. 

Meanwhile, researchers who are not 
directly interested in methodological 
issues in interviewing cannot afford 
to ignore them if they are conducting 
research with retarded persons that 
entails asking questions. Again, 
the literature on survey research, 
as well as the present monograph, 
can provide guidelines to follow in 
constructing instruments. The re- 
searcher sensitive to the issues 
will probably want to build checks 
on response validity into whatever 
instruments are developed. We are 
confident that researchers will 
detect many other leads in this 
monograph that we have not mentioned 
here . 

How to Interview 
Mentally Retarded Persons 

Based on our experience as well 
as our data, wc would make the fol- 
lowing recommendations to persons 
seeking information from mentally 
retarded individuals through survey 
research. (The reader is also 
referred back to Chapter 4 for more 
about the logistics of interviewing.) 

9 



PREPARATION 

Invest a substantial amount of time 
in preparation before any "real" 
interviews are conducted. 

It is not unusual in the mental 
retardation field or in any other 
area of research for researchers to 
"slap together" an interview or 
questionnaire. There is not a 
strong tradition in survey research, 
as there is in other areas of 
measurement such as testing, of 
establishing the reliability and 
validity of instruments. We have 
"slapped together" interview sched- 
ules for mentally retarded persons 
ourselves. While we pilot tested 
many questions used in the present 
study, our efforts could still be 
characterized as informal at best. 
Granted, we labored individually and 
in work groups for hours deciding 
what we wanted to ask and how to 
phrase and structure questions on 
those topics. We went through many 
draft versions of these interviews 
before we were satisfied. But, most 
of our planning was based on armchair 
thinking, on an attempt to imagine 
all the problems a question might 
present to a mentally retarded per- 
son. There were still numerous 
occasions in which we found, while 
administering interviews, that we 
had not anticipated everything, that 
we were getting bad information be- 
cause our question was bad. To give 
just one example, we found that 
institutionalized persons often had 
difficulty with questions about their 
activities due to the fact that they 
were not sure whether we meant to 
ask them about what they did at the 
institution, what they did at home, 
or both. A few interviewees ex- 
plicitly asked which we meant, but 
mott others simply answered, and 
theye is no way to tell how they 
interpreted the questions. If we 
had wanted to know their extent of 
involvement in, for example, home- 
making activities in the cottage at 
the institution, we would have had/ 
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to add the phrase "in your cottage" 
or the like to each question. We 
indeed did just that once we recog- 
nized that our initial questions were 
ambiguous, but by then we had col- 
lected data from whole samples and 
had no way to be sure whether their 
answers pertained to activities at 
the institution, at home, or both. 

We would recommend starting with a 
serious "armchair" process of de- 
signing what appear to be simple 
\and unambiguous question's about 
whatever topics are of interest. 
However,' we would then recommend 
moving from the armchair to the 
field to do extensive piloting of 
the draft interview schedules with 
real retarded persons similar to 
tho^e to be interviewed in the full- 
fledged study. No matter how well 
one anticipate exceptions, misinter- 
pretations, and ambiguities, one is 
certain to detect more during the 
piloting process. If it is feasible, 
we would recommend conducting 
parallel interviews with attendants, 
parents or houseparents, or obtaining 
validating data from files and 
records so that the pilot testing 
process yields information about the 
validity of answers, beyond what can 
be obtained by seeing bow inter- 
viewees are interpreting questions. 
Because interviewing retarded persons 
^is particularly difficult, this kind 
of preparatory work appears to be 
crucial . 

QUESTION DESIGN 

Draw on the successes and failures 
of the present study in designing 
questions. 

To be as brief as possible here, 
that would include paying careful 
attention to question wording and 
using question formats that are 
likely to optimize responsiveness 
and validity of response (see Chapter 
8 for more details) . We would rec- 
ommend an interview schedule that 
is highly structured in basic design. 



with open-ended questions used as 
follow-up questions in areas where 
more clarification is desired from 
more verbal respondents. We would 
recommend heavy use of either-or 
questions and multiple choice ques- 
tions with discrete rather than 
quantitative options. If severely 
retarded persons are to be interviewed, 
such questions should, when possible, 
be accompanied by pictures. Simple 
open-ended questions calling for one 
piece of information rather than an 
enumeration would also be appropriate, 
although it must he recognized that 
many less/ verbal persons will not be 
able to answer them, even if they 
concern very basic facts. By using 
structured questions as the basis for 
the interview schedule, one can hope 
to obtain at least some information 
from virtually all persons in the 
sample. This information can l^e 
supplemented with elaborative and 
illustrative material gained from 
open-ended questions that only some 
respondents can answer. We would 
also recommend asking as few ques- 
tions as possible that require con- 
cepts of number, time, quantity, 
monetary value, history, and the like. 

Based on piloting, one can anticipate 
problems that interviewees are likely 
to have and design, in advance, 
paraphrases or alternative phrasing 
of questions to be used if inter- 
viewees either cannot understand or 
ask for clarification of the original 
question. Alternatively, the deci- 
sion can be made to ask questions 
only in one standard phrasing and 
simply live with any missing data 
that result. The advantage of the 
first alternative is that one can 
hope to gather information from a 
larger portion of the sample under 
study; the risk is that the alterna- 
tive way of asking for the informa- 
tion may change responses. The ir.-^re 
like the original question the 
clarifying question is, the less that 
risk will be. Indeed, our analysis 
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of responsiveness suggests that 
simply repeating the question in the 
same form will increase responsive- 
ness, with no alteration of the 
meaning of the question. Thus, such 
repetitions are clearly recommended. 
Still, there are inevitably some who 
cannot understand the question even 
when it is repeated. 

Similarly^ .probes and follow-up 
questions generally should be 
planned in advance. A sensitive 
interviewer can do wonders in pur-- 
suing an answer and drawing a person 
out; as a re-nlt, we have a reluc- 
tance to ad ze total standardiza- 
tion. However, our concern with the 
potential impacts of question format 
and phrasing on responses makes it 
necessary. Our recommendation that 
interview schedules be layered, with 
structured questions followed up by 
open-ended ones, helps in this re- 
gard. The structured questions would 
provide exactly the same stimuli to 
all respondents. Then to gain 
further detail from more verbal 
siabjects, the interviewer might be 
given discretion to pursue interest- 
ing responses, as long as that inter- 
viewer is sensitive to the effects 
of question format and wording and 
will not lead the respondent. 

We have recommended that yes-no 
questions be avoided if at all pos- 
sible, and will stick with that rec- 
ommendation. However, if they are 
used, we would strongly urge that 
checks for the operation of acquies- 
cence be built into the interviewing 
process. Our data on acquiescence 
have suggested that it is not just a 
matter of isolating a few acquies- 
cers; we have found instead that 
most retarded persons will acquiesce 
to some extent at some time. Still, 
the phenomenon is more pronounced 
for some persons. One possible check 
is to scatter oppositely worded 
questio-.s of the type we have used 
throughout the interview schedule 
and determine how many times respon- 



dents contradict themselves by 
answering »'yes" to both questions in 
a pair. The other strategy we used, 
asking questions that should properly 
be answered "no," appears to be 
equally appropriate. One might wish 
to develop a formal acquiescence 
scale using one or both of these 
approaches, testing it out in advance 
to make sure it is reliable and valid. 
Even^an informal check, however, 
would permit detection of frequent 
acquiescers. We ar^ not yet in a 
position to suggest an explicit - 
criterion for defining. frequent 
acquiescence. it would-be up to the 
researchers involved to Establish 
such a criterion and then^^iise it to 
eliminate acquiescers from\he usable 
sample. However, because checks for 
acquiescence would inevitably mean 
disqualifying some responses, we 
continue to think that 'it is best to 
avoid asking yes-no questions if at 
all possible. 

HEARING FROM SEVERELY AND PROFOUNDLY 
RETARDED PERSONS 

Use alternative ways of obtaining 
information about and from more 
severely retarded persons. 

While we have not always found 
strong relationships between IQ and 
_the. interview measures we have 
examined here, we have generally 
found more problems interviewing per- 
sons in lower IQ ranges. Our one 
attempt to interview profoundly re- 
\tarded persons, done simply to docu- 
\ment the obvious, was an almost 
total failure. Severely retarded 
persons, while less predictable, also 
have relative difficulty answering 
verbal questions. For them, v found 
the pictorial either-or' format ^.^arti- 
cularly promising, but we must still 
conclude that they have limited 
capacities to respond and that they 
are particularly susceptible to re- 
sponse biases such as acquiescence 
and a preference for the last option 
in an either-or question. The 
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two-layered interview schedule 
approach we have advocated here 
would perhaps allow one to obtain 
answers to structured questions from 
most severely retarded persons, but 
few of them would be able to elabo- 
rate on their answers in response to 
open-ended follow-up questions. 
Thus, there appears to be no way to 
avoid underrepresenting the per- 
spectives of more severely retarded 
persons in survey research. 

There are alternatives. One which 
has been used before is to obtain 
the desired information from a 
parent/ attendant, houseparent, or 
other respresentative of the retard- 
ed person. However, there are 
limitations to this approach. We 
have at times encountered cases in 
which attendants or parents and 
retarded respondents simply disagreed 
about something, where one cannot 
explain discrepancy between their 
answers as a matter of response bias 
on the 'part of the retarded respon- 
dents. We have also found numerous 
instances in which attendants and 
parents have surprised us by not 
knowing something that we assumed 
they would know about the retarded 
persons they interact with. 

On the other hand, we feel that 
relying on /Such persons as informants 
is better than simply writing off 
severely and profoundly retarded 
persons. Indeed, even when mildly 
and moderately retarded persons are 
interviewed, we would recommend 
interviewing their "significant 
others" as well, partly as a check 
on the validity of certain responses, 
and partly to obtain information that 
most retarded persons have difficulty ' 
providing themselves (e.g., informa- 
ytion about history, involvement with 
^service agencies, financial matters, 
and amounts of time involved in 
various activities) . Parents and 
care providers are especially valu- 
able as sources of factual informa- 
tion; whether they can provide 



information about retarded persons* 
perspectives and opinions is another 
matter. We would urge that severely 
retarded persons still be given an 
opportunity to speak for themselves, 
particularly about their opinions, 
even when nonretarded informants 
must be the primary source of in- 
formation about them. 

Another alternative to interviewing 
low verbal individuals is behavioral 
observation. Instead of asking them 
about daily activities, one might 
observe them through time sampling 
in the residential setting. Instead 
of asking about satisfaction with the 
residential setting or with service 
providers,, one could potentially 
observe affective responses to ser- 
vice providers or to events in the 
residential setting. We are not 
claiming that this would be easy, for 
behavioral observation is time-con- 
suming and difficult. However, that 
might be the best way, coupled with 
interviews of informants knowledge- 
able about the retarded person, to 
approximate an understanding of the 
needs, perspectives, and circum- 
stances of persons with limited or non 
existent verbal skills. 

To be sure, the data would not be 
comparable to data obtained from more 
verbal individuals through interviews. 
Thus, we have no easy solution to the 
problem of obtaining comparable data 
from the entire range of retarded 
persons. VJhen previous researchers 
(e.g.. Bell, 1976) have relied on 
informants to answer for low-IQ 
persons but have ' obtained answers 
directly from higher-IQ persons, 
there is simply no way to tell 
whether any resulting differences 
between high and low-IQ persons are 
due to real differences in their 
lives or due to the difference 
in data sources. This same quandary 
would be true any time the source 
of information for one group is 
different from the source of infor- 
mation for another. However, it 
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would be possible^ to interview both 
clients and nonretarded informants 
in the high IQ range, and use the 
agreement between theit responses as 
a way of roughly estimating the 
degree to which responses given by 
nonretarded informants and sever:=*iy 
retarded persons might agree. if 
agreement is high, One would feel 
more comfortable combining nonre- 
tarded informants ' data for low~IQ 
persons with retarded informants* 
data for high-IQ ones. 

We should perhaps make some mention 
of the nonverbal communication 
systems now used with language- 
deficient retarded persons, in our 
opinion, while such systems are per- 
mitting great advances in the com- 
munication training of severely and 
profoundly retarded persons, the 
time has not yet arrived when they 
can be used to collect da":a from 
samples of retarded persons on a 
widespread basis. The major problem 
is that even if interviewers could ' 
be trained to administer the inter- 
view in a nonverbal communication 
system, what nonverbal commxinication 
system would be used? There is no 
standardization at this point, as 
there would be if one wanted to 
interview deaf persons and could use 
Ameslan as the vehicle. While 
Ameslan has been used with some 
retarded persons, many :.^ther systems, 
including sign languages developed 
especially for use within a particu- 
lar facility, are too provincial to 
be used in a large-scale interviewing 
project. It might be possible to 
enlist the help of facility staff to 
"translate" interview questions inco 
a local nonverbal communication 
system. However, one would then 
have to guard against the possibility 
that respondents would react dif- 
ferently to a staff interviewer 
than to an cutside interviewer. All 
things considered, then, these 
systems do >iot offer much hope at 
the present time, except, of course. 



as a means for facility staff to 
gather input from their own residents 
for program planning purposes. 

Implications of the Study 

for the Mental Retardation Field 

THE FEASIBILITY OF A. NATIONAJ. POLLING 

This project was. inspired by the 
President's Committee on Mental 
Retardation, which wanted to know 
whether or not a periodic national 
polling of retarded citizens would 
be feasible as a means of obtaining 
consumer input, in light of our 
findings, the answer to this question 
depends on the form this proposed 
polling takes. A comprehensive sur- 
vey of mentally retarded consumers, 
comprehensive in the sense of involv- 
ing' the whole range of mentally 
retarded persons, and comprehensive 
in the sense of covering a broad 
spectrum of topics relevant to the 
lives of the retarded, appears not 
to be feasible. Very sin;/*'', too 
many retarded persons wou. lave too 
much difficulty providing valid in- 
formation about too many topics- 
However, less inclusive surveys, 
cortstructed and administered with our 
findings in mind, could provide PCMR 
and others with valuable information 
about the needs, circumstances, and 
attitudes of particular groups of 
retarded consumers. 

The data suggest that a standardized, 
comprehensive, national, survey of 
mentally retarded persons is imprac- 
tical on two major grounds. First, 
it is clear that standardized inter- 
viewing of the entire spectrum of 
mentally retarded consumers is not 
possible. Our experience indicates 
that interviewing profoundly retarded 
persons is clearly impracticable, 
and that the population of severely 
retarded consumers includes many 
people who are unresponsive or pro- 
vide responses of very low validity. 
Further, even among the mildly and 
moderately retarded, many persons 
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give responses to interview ques- 
tions which are largely invalid, 
serious attempts to interview re- 
tarded 'persons must include validity 
checks which screen out the answers 
of people who consistently display 
response biases. Thus, survey re- 
search with retarded consumers 
necessarily involves a selective 
sample of more verbally proficient 
and intelligent persons, and the 
information gained might be quite 
unrepresentative of the entire 
national population. 

Secondly, the data presented in the 
preceding chapters indicate that 
there are serious restrictions on the 
types of information which can be 
obtained from interviewers with men- 
tally retarded respondents. The only 
question formats which generated 
information of reasonable validity 
in our project were either-or ques- 
tions, multiple choice questions 
offering discrete rather than quanti- 
tative response options, and open- 
ended questions targeted at discrete 
bits of factual information. Re- 
striction of an interview schedule 
to these formats limits the types of 
information that can be sought. 
Further / we have found that certain 
types of content areas are especially 
problematic; specifically, questions 
using such concepts as time, number, 
quantity, monetary value, history, 
and so on are difficult for mentally, 
retarded persons and tend not to 
generate meaningful answers. Thus, 
we cannot conduct a polling of re- 
tarded consumers whj.ch is comprehen- 
sive with regard to topics included. 

our study indicated unambiguously that 
retarded persons would have too much 
difficulty providing valid informa- 
tion for a comprehensive national 
survey to be feasible. We have not 
discussed some of the formidable 
problems in constituting a national 
sample; distinguishing between the 
identified population and the "actual" 
population would be difficult, and 



gaining access to even the identified 
population, given current safeguards 
protecting the confidentiality of 
client records, would be virtually 
impossible. 

However, even if a comprehensive 
national survey appears to be a poor 
means of information gathering, the 
goal of hearing from retarded individ- 
uals remains an important and viable 
one. VJe would be enormously discour- 
aged if our study caused people who 
work with the retarded! to give up; to 
stop trying to hear from their 
clients regarding crucial issues in 
their lives. Rather, implicit in our i 
statement of what we cannot do is a i 
host of information regarding what we 
can do, and it is to the possibili- 
ties for communicating with mentally 
retarded consumers that we turn next. 

THE NEED FOR PERSISTENCE 

Even though we have had, at times, 
to be pessimistic about interviewing 
retarded persons,, our data do indicate 
that one can interview some of the 
people some of the time. It is our 
hope that this monograph, rather than 
discouraging people from attempting to 
gather input directly from mentally 
retarded cqrisumers, will encourage them 
to do so in a methodologically sound 
way: As we see it, the most serious 
problems we have encountered are those 
that involve systematic response biases. 
Acquiescence is the single most 
serious problem we have encountered, 
and efforts to guard against it must 
certainly /i)e made. However ; when 
there is no systematic response 
effect operating, the threat to 
validity is usually not as great. 
Granted, there still may be error in 
the data of substantial size, but 
the picture one obtains of a group 
through one questioning technique 
may not differ greatly from the 
picture one obtains from an alterna- 
tive technique. At other times, we 
have found retarded- persons able to 
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answer and able to answer with valid 
information. Furthermore, we have 
suggested some ways of optimizing 
responsiveness and response validity 
which we hope will be useful to 
others. We have assumed and continue 
to assume that because communication 
is a two-way street, communication 
with retarded persons can be improved 
if interviewers improve their tech- 
niques of soliciting information. 

However, our experience with retarded 
persons has instilled in us a com- 
mitment more basic than our commit- 
ment to increased methodological 
sophistication. That commitment is 
to an increased sense of humanity 
and sensitivity in our dealings with 
mentally retarded persons. Our 
empirical data somehow miss a vital 
point about our interactions with 
the retarded: namely, that nearly 
every one of our respondents, even 
low verbal individuals, had some- 
thing to say. Repeatedly, people 
we interviewed told or showed us 
that being interviewed was for them 
a rare, exciting, and gratifying 
experience - Perhaps this is a -commen- 
tary on the fact that their views 
are often ignored, or at least that 
nonretarded people rarely sit down 
for any length of time with them 
individually and ask them to talk 
about themselves. For the inter- 
viewer who enters into the task with 
a genuine regard for retarded people 
^and a real desire to learn about 
them, the interviewing ptocess is 
equally rewarding. In fact, we 
developed a strong sense that in 
those interviews which did not elicit 
much information, the failure result- 
ed not because the mentally retarded 
interviewees did not have the right 
answers, but because we did not ask 
the right questions. Our structured 
interview schedules not only taxed 
the communication skills of our 
interviewees, but often failed to 
give them a genuine opportunity to 
speak their minds. A more flexible 



approach, tailored to the capabili- 
ties and interests of the individual 
client, frequently produces a less 
rigorous but richer interaction. At 
least we found this to be the case 
in our more free-wheeling interviews 
with adults living in the community. 

Thus, in its entirety, our study 
provides a set of more or less specif- 
ic recommendations for the field of 
mental retardation. First, the idea 
of a comprehensive national polling 
of retarded consumers is probably not 
practical, at least given our current 
knowledge. However, a segment of the 
retarded population can respond to 
traditional, standardized interview 
techniques, and can give valid in- 
formation if interviewed within the 
constraints "discussed in the previous 
section. We suggest that workers in 
the field intensify efforts ro intei 
view, with methodological rigor, 
particular groups of mentally retard- 
ed consumers of services. in addi- 
tion, even when formal interviewing 
is no't possible, , given the person or 
the topic, retarde^i persons may still 
have* a great deal to say. For those 
individuals and ropic areas not 
"amenable to formal, standardized 
interviews, we recommend careful 
efforts to communicate in less struc- 
tured, less formal interviews, 
tailored to the needs and capabili- 
ties of the individual client. Our 
project generated some useful guide- 
lines as to how this process of in- 
formal communication can be made 
more efficient and can generate more 
valid information. It is to the 
guidelines that we can turn next. 

ISSUES FOR CONSIDERATION 
BY PRACTITIONERS 

For the most part, this mono- 
graph has been directed toward re- 
searchers and others who conduct 
formal interviews with retarded per- 
sons for data-gathering purposes. 
Yet there is no great difference. 
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except for degree of formality, 
between a survey research project 
and the kinds of informal question- 
ing and interviewing that case - 
workers, counselors, and other pro- 
fessionals do daily with retarded 
clients. All major legislation 
affecting retarded persons today, 
the Education for All Handicapped 
Children Act and the 1978 rehabili- 
tation amendments, including the 
developmental disabilities sections, 
calls for the development of indivi- 
dual program plans. Moreover, they 
call for the involvement of clients 
or students, or their representatives, 
in the development of such plans. 
Beyond the individual program plan- 
ning process, professionals who work 
with the retarded frequently 
need to obtain information from 
theih and to interact with them 
verbally in order to carry out treat- 
ment and training efforts and to 
evaluate individual progress and 
overall programs. 

We view our findings as just as 
relevant to the practitioner as to 
the researcher. Take, for example, 
the natter of yes-no questions. 
Everyone who talks to a young child 
or a retarded person with limited 
verbal skills slips into using them. 
Yet our findings clearly indicate 
that retarded persons are highly 
likely to say "yes" to such ques- / 
tions, whether "yes" is the appro- ^ 
priate answer or not, and that the^ 
answers one obtains to tv/o yes-no/ 
questions that are oppositely woifded 
are quite likely to be discrepant as 
a result. W$ think practitiohers 
should be alert to the possibility 
of acquiescence in their discussions ■ 
with retarded clients, particularly 
if important decisions are going to 
be based on whether the client says 
"yes" or, "no." One might want to 
consciously attempt to re-ask a . 
yes-no question later in the inter- 
view with \the wording reversed (for 
example, ajter asking, "Do you think 



the car wash would be a good place 
to work?" ask, "Do you think the car 
wash would be a bdd place to work?")- 
If the answers- are both "yes," one 
at least knows that further question- 
ing will be necessary to clarify what 
the client really does want. Alter- 
natively,. as we have suggested here, 
an either-or question ("Do you think 
the car wash is a good place to work 
or a bad place to work?") might be a 
better way of obtaining a valid an- 
swer. 

The practitioner might benefit in 
other ways from being more sensitive 
to the effects of question wording 
and format. Use of the simplest 
wordings possible is a starting 
point. Again, it is useful to ask 
the same question in two or three • 
ways to check on client comprehension. 
Wording is particularly problematic 
when one is asking about service 
agencies. Our studies, for example, 
have suggested that clients have 
varied understanding of what vocation- 
al rehabilitation is. If asked 
whether they have been a vocational 
rehabilitation client, retarded per- 
sons may not understand that wording, 
but may recognize "voc rehab," or 
"TRC," or another local name for the 
rehabilitation agency. (We have some 
evidence to suggest that they may 
be even more likely to talk about 
their experiences with an agency if 
you are able to name the counselor 
they actually had. Many of our 
respondents related to a service 
agency in terms of a specific staff 
member, sometimes not being able to 
identify the agency or what its 
functions as a whole was called or was 
all about.) 

As we have also suggested, using 
examples to make a question more 
concrete: may backfire by changing 
the meaning of the question or by 
prompting mention of the examples in 
answers. Generally, then, one would 
want to be careful not to lead the 
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client toward a response that he or 
she might otherwise not givel 

When the same information is being 
sought from several clients (for 
example, in a program evaluation 
survey) , the same standardization of 
questions that we would recommend 
for researchers should be used. One 
often cannot predict how much a 
slight change in wording might affect 
the answers that are given. On the 
other h^nd, we are quite confident 
that if some clients are asked a 
yes-no question and others are asked 
an either-or question on the sane 
topic, their answers are likely to 
differ. This clearly points to the 
need for careful planning in design- 
ing the questions to be used and ad- 
herence to those questions in actual- 
ly conducting the interviews. The 
interviewer might still be given 
latitude to pursue interesting or 
confusing answers to the standard 
questions. To be sure, the prac- 
titioner does more informal types of 
data-gathering than the researcher, 
and does not have as much concern 
with methodological issues. But 
this does not mean that the practi- 
tioner does not need to be concerned 
with the validity of the answers 
that; are given. 

Finally, a note. to teachers\ and 
trainers of the mentally retarded is 
in order. Our findings are highly 
relevant to teaching communication 
skills to retarded persons. Overall, 
they suggest that many retarded 
persons are in need of training which 
would better equip them to understand 
and answer questions, not only in 
interview situations but in everyday 
exchanges with others. Too often, 
we feel, language and communication 
training emphasizes the formal rules 
of language but does not go far 
enough to help retarded students use 
language appropriately in different 
social contexts. Retarded persons 
need explicit training in answering 



questions appropriately so that their 
answers meet the demands of the 
questions (e.g., so that they answer 
an either-or question with one of the 
two options rather than by saying 
"yes"* as many of our subjects did) . 
Beyond that, as our findings clearly 
suggest, they may nee 4 guidance in 
answering questions accurately, es- 
pecially questions about basic per- 
sonal information. 

More specif ically , we would offer the 
following suggestions for inclusion 
in communication training prograifis. 
First, concentrate on teaching re- 
tarded persons to answer questions 
appropriately and to avoid answers 
that are unintelligible, irrelevant 
to the question, or so vague that 
they are uninf ormative . Our data on 
responsiveness to questions suggests 
a developmental hierarchy of types 
of questions, progressing from yes-no 
questions and questions involving 
pictures that can be answered by 
pointing, to either-or questions, to , 
multiple choice questions, and final- 
ly\ to open-ended questions. Although 
we have not emphasized it here, there 
are\ also levels of difficulty within 
the category of open-ended questions. 
For example, in previous research 
(Sigelman & Werder, 1975), retarded 
adults were more responsive to 
what-questions asking them to name 
or describe something than they were 
to why-questions and questions that 
asked them to consider hypotheti- 
cal situations (e.g., "What would 
make things better for you where you 
live?"). \ Training might involve 
asking , students progressively more 
difficult types of questions while 
giving them feedback and instruction 
as to what the question is demanding. 
Once the student can supply minimally 
appropriate responses, the trainer 
might encourage them to provide more 
complete responses and elaborate on 
their answers. \ For instance, if t he 
question is "Do you have a job?" a 
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minimal response might be "yes," a 
better response might be "Yes, I have 
a job," and an even better one might 
be "Yes, I work part-time at a cafe- 
teria." 

In addition, training should be 
directed toward obtaining valid 
answers, not just answers. Two 
points are particularly worth empha- 
sizing. First, as we have shown, 
many retarded persons are not' able to 
-answer basic questions about them- 
selves such as full name, birthdate , 
address, and so on. For some, the 
problem* is answering an open-ended 
question at all, due to limited ver- 
bal skills. However, we feel that 
even those with limited verbal skills 
could be taught to respond by rote 
to a set of basic questions about 
themselves. For others, the diffi- 
culty lies not in ansv^ering but in 
answering accurately, and these in- 
dividuals also need training. We 
feel strongly that retarded persons 
should be able (for example, in an 
emergency) to .tell people such 
things as who they are, where they 
live, their age or birthday, their 
telephone number, and so on. 

Secondly, we feel that acquiescence 
and possibly other response biases 
should be given attention in com- 
munication training programs. One 
simple way to do this is to have 
students practice answering questions 
about concrete objects and materials 
or about verifiable facts in their 
immediate environments (e.g., "Is' 
your shirt blue?" "Am I your fa- 
ther?") . This allows the instructor 
to provide feedback about the accura- 
cy of answers that are given as well 
as their form. Quite obviously, re- 
tarded persons can get into trouble 
in the "real world" if they continual- 
ly say "yes" when they are asked 
questions calling for a "yes" or "no" 
answer. Rosen, dark ^d Kivitr: (1977) 
have~'explbr"ed this'acquie scence prob- 
lem in the larger context of compli- 
ance with requests and have found. 



for example, that many retarded per- 
sons will agree to take a pill not 
knowing anything about what the_ pill 
contains. In every area of their 
existence, fr6m sex to individual 
program planning, retarded persons 
must learn to avoid exploitation and 
assert themselves. At least part of 
this larger mission can be accom- 
plished by teaching them not to say 
"yes" without thinking about the 
answer. 

CONCLUDING REMARKS 

We would like to leave our readers 
with a final message: retarded people 
must finally be treated as people. 
For service providers and researchers 
alike, this concern with basic rights 
includes an obligation to obtain in- 
formation concerning the perspectives 
of the retarded. Too often we have 
not bothered to ask retarded people 
themselves what is going on and what 
they think of it. We sometimes 
assume too readily that a residential 

program or an educational activity 
or a service system that looks good 
to us must look good as well to the 
retarded clients it serves. We have 
relied too often on the opinions of 
"significant others" to determine 
what retarded individuals need or 
want. Although data obtained direct- 
ly from retarded persons through 
interviews cannot totally replace 
other sources of information, it 
should be one of the ways in which we 
attempt to "know" retarded persons. 
If questions are hastily designed 
without attention to measurement 
issues of the kind discussed here, 
the gains will be few. Indeed, the 
data collected might easily be mis- 
leading; for example, if program 
evaluators 'ask residents of a fa- 
cility questions like "Are you satis- 
fied with the training hare?" they 
thereby assure themselves of a favor- 
able evaluation report. However, if 
care is taken in deciding how to 
seek information, the gains can be 
great. Furthermore, our data clarify 

17 



217 



an important point: even informal 
interactions with mentally retarded 
persons must be conducted wi,th sensi- 
tivity to the level of their communi- 
cation skills. A well-intentioned 
effort to "make things easier" for a 
mentally retarded person by phrasing - 
inquiries as yes-no questions, can 
have the effect of destroying that 
person's opportunity to communicate. 

In closing, we cite two of the less- 
structured interviews we conducted' 
with mildly retarded^ Relatively 
verbal adults living in the community, 
in which we asked them to reflect, on 
the implications of being retarded, 
(Virtually all of these adults denied 
being retarded, but had much to say 
about. the implications of being con- 
sidered retarded.) We believe these 
transcripts are an excellent illus-- 
tration of the value of asking re- ^ 
tarded persons to speak for themselves. 

r 

Interview i 

Q: W^at bad things happen to you 
'when people think you are re- 
tarded? 

A: Not too much has happened to me 
bad. They been pickin on me. 
They say, "You can't do that 
because you're mentally re- 
tarded. You can't do this." 
They ^ust sort of push me back 
when I want to do something and 
say, "Oh, you can't come with 
us. We don't want you." 'They 
just push you back into a cor- 
ner. "She can't do thisl She 
doesn't know what she's doing." 

Q: What good things happen to you 
when people think you're re- 
tarded? 

A: Oh, they seem to want to take- . 
me out to : - erent places like 
football sc?- . .. * and picnics 
- and going out Lw ui o falls and 
spending the day out there. 
And going on all day trips. 
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Q: Does anything else good happen? 
A: No. 

Q: 0-KI, what would it be like 'for 
you if people stopped thinking 
you were- retarded? 

A: It'd be pretty good. We could 

probably get along a lot better - 
if people didn't think of us 
as being retarded and handi- 
capped. We'd probably make it 
out on our own- But see, the 
rest of the people are just 
pushing you like in one certain 
square and saying, "That's 
where you can be; you can't 
come over here." Because I 've 
had that all my life. 

Qj HOW do you think it would be 
different? What would you be 
doing differently if people 
hadn't done that to you? 

A: Uh, I think I'd have higher 
paying jobs and be able to go 
to restricted movies; not that 
we want to, but you know, if 
you wanted to, be able to do 
it and not having to say , "We 
don't allow you in here." 

Q: When you say people telling you 
where to go , who do you mean 
exactly? 

A: Generally everybody that thinks 
that . . . well, like when I 
' was going to school we had a 
special section for special 
education. And everybody 
that went by there would call 
us names or throw things at us 
and say, "We don't want to be 
with you; you're mentally re- 
tarded . You 're crazy . " 
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Interview II 

Q: How do most people treat you 
when they think you are retard- 
ed? 

A: Bad. 

Q: What kinds of things do they do? 

A: Yell at you, like when I'm with 
friends (unintelligible) , 
things like that. Makes you 
feeil about that big (gesturing 
to show tiny size.) 

Q: Uh, huh. Does everybody do it, 
or does it seem like . . . 

A: I'm smart enough -to-knQw_the_, -' 
difference in people. Some of 
'em treat me real nice; like 
everybody else. Then some of 
'em don't. 

Q: Does it bother you when people 
think you are retarded? 

A: Yes, it does. I feel like I'm 
that big (gestured small size). 
It scares me a little bit. 

Q: Are there things that you would ' 
like to do, but you cannot do 
them because somebody thinks ' 
you are retarded? 

A: Yes. ^ 

Q: What are those things? 

A: Well, like I've worked in a 
nursing home and they treat 
you real bad there. When they 
thought I was retarded. 

Q : Oh really? 

A: Treat me real bad. And they 
made me lose my job just for 
that. 

Q: How did they treat you? What 
did they do exactly? 

A: I tried to help them and they 
didn't want to be helped. And 
said I was retarded and all 
that. . They were real ugly. 



Q: Were these the residents at the 
nursing home or the othei/ 
people who worked there?/ 



A: Other people. The employees. 
That's the ones that you have 
a problem with when you go to 
work somewhere like thal^. The 
employees. ; 

Q; Are there any other things that 
you'd like to do but can't 
because somebody thinks you're 
retarded? 

A: I don't know. They start making 
fun of you or something. Like 

-yester.day__Me/were on a bus 

and this old lady was making 
fun of my husband. And I didn't 
like that at all. I was getting 
ready to tell her where to go I 



Q; I imagine so. 
blue? 



Just out of the 



A: Yeah, out of the blue. Look 
at you, and then make fun of 
you, and I don't think that's 
really right. It really does 
make me mad. It (unintelli- 
gible) you know, with our 
rights. You know we have our 
rights now.. It's law we have 
our rights. If there's any- 
thing you want to, \ih, through 
the law. I knew that long ago. 
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Interview Schedule: Cormnunity Children 



1. What is your full name? 

(Probe: "What is your last name?") 

2. How do you spell your name? 

(Probe: "How do you spell your last/first name?") 

3. Please write your name for me on this paper. 

4. How old are ycu? 

5. What is your address at home? 

6. Are you usually happy or sad? 

7. How many friends do you have: a lot, some, not many, or none? 

8. When you're not at school, are you usually with other people? 

9. Do you play any sports? 

10. (If yes to "play sports", ask) What (other) sports do you play? 

11. Are you usually happy? 

12. Do you watch TV? 

13. Do you listen to the radio or record player? 

14. Do you go out to eat at restaurants or. cafes? 

15. Do you go to church? \ 

16. When you're not at school, are you usually by yourself or with other 
people? 

17. Do you know how to read books? 

18. Is anybody at school teaching you about reading now? 

19. Do you know how to write sentences? 

20. Is anybody at school teaching you about writing now? 

21. Do you know how to count money? 

22. Is anybody at school teaching you. about money now? 

23. Are you usually sad or happy? 




24. When you're not at school r are you usually by yourself? 

25. Here are some ways people get to school. (Point as you say): They 
take the bus, somebody drivGS them to school in a car, they walk, or 
they ride to school on their bicycles, which way do you get to school 
most days? Point to the picture. 

(Interviewer may probe with "Most days, how do you get to school?") 

26. Do you live in a house, an apartment building, a trailer house, or a 
duplex? 

27. Are you usually sad? 

28. Counting you, how many people live in your house riqht now? 
(Interviewer probe, if necessary, with " How many is that, counting you?" 

or answer "Yes" if asked if we want a number.) 

29. When you're not at school, are you usually with other people or by 
yourself? 

30. If you had one wish, what would you wish for? 

31. Afost of the time, is it up to you to decide what time you go to bed? 

32. What time does school start every day? 

33. Afost days, does somebody else tell you what to wear to school, or do 
you decide? 

(Probe, if necessary, with" "Most days?") * 

34. Is anybody at school teaching you about cooking now? 

35. This picture (point) shows a boy/girl who is happy, and this one 
(point) shows a boy/girl who is sad. Which picture shows how you 
usually feel? Point to the picture. 

36. Here are some pictures of different kinds of places people live in. 
(Point to each as you say) Here is a duplex, a house, a trailer house, 
and an apartment building. Which picture shows the kind of place you 
live in? Point to the picture. 

37. Most days, how do you get to school? 
(Interviewer probe, if necessary, with Afost days.") 

38. What things would you really like to learn in school? 

(Probe one time only with: Wliat other things would you really like 
to learn in school?) 

39. This picture (point) shows a boy/girl who is by himself /herself , and 
this one (point) shews a boy/girl who is with other people. When you're 
not at school, which boy/girl is most like you? Point to the picture. 

40. Is anybody at school, teaching you about numbers now? 



A. 2 

22^ 



41. 



What time do you usually go to bed at night? 
(Probe, if necessary, with: "Usually.") 



42. Most days, is it up to you to decide what to vrear to school, or does 
somebody else tell you what to wear? 

(Probe, if necessary, with: "Most days.'*) 

43. Are you allowed to go places by yourself, or does somebody always take 
you? 

44. This picture (point) shows a boy/girl who is sad, and this one (point) 
shows a boy/girl who is happy. Which picture shows how you usually 
feel? Point to the picture. 

45. Most of the time, does somebody else tell you what time to go to bed? 
(Probe, if necessary, with: "Most of the time.") 

46. Do you live in a duplex, a house, a trailer house, or an apartment 
building? 

(Probe, ii nscessary, with: "Which one: duplex, a house, a trailer 
house, or an apartment?") 

47. Is anybody at school teaching you about keeping house now? 

48. Most days, do you get to school on the bus, in a car, by walking," or 
riding a bicycle? 

. (Probe, if necessary, .with: "Most days," Also may have to probe: 
"Which one: the bus, a car, walking, or a. bicycle?") 

49. Is anybody at school teaching you about getting jobs now? ' 

50. Are you ever allowed to go places by yourself without somebody taking 
you? 

Here are some pictures of different kinds of chores some people do. I 
want you to tell me if you do any of these chores at home- 

51. Do you set the table? 

(Probe, if necessary, with: "Is that a lot,. some or not much?") 

52. Do you do dishes? 

53. Do you clean the floor? 

54. Do you dust furniture? 

55. Do you make beds? 

(If yes:) Do you make beds a lot, some or not much? 

(Probe, if necessary, with: "Is that a lot, some or not much?"). 

56. Do you pick up stuff around the house? 

(If yes:) Do you pick up stuff around the house a lot, some, or not 
much? 

(Probe, if necessary, with: "Is that a lot, some, or not much?") 

A.3 
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57. Do you cook on the stove? 



58. Do you make sandwiches? 

59. .Do you take out the trash? 

60. Do you work outside in the yard? 

61. Here are some pictures of different kinds of places people live in. 
(Point to each as you say:) Here is a house, an apartment building, 
a trailer house, and a duplex. Which picture shows the kind of place 
you live in? Point to the picture. 

62. Are the neighbor kids very friendly, just so-so, or not friendly to 
you? 

(Interviewer probe, of necessary, with: "Which one? very friendly, or 
so-so, or not friendly?") 

63. This picture (point) shows a boy/girl who is with other people, and 
this one (point) shows a boy/girl who is by himself/herself. When 
you're not at school, which boy/girl is most like you? Point to the 
picture . ^ 
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Interview Schedule: Institutionalized Adults, Form A 

1. What is your full name? 

2. How do you spell your name? 

3. Ploase write your name for me on this paper. (Use back of form) 

4. What month is your birthday? 

5. What day is your birthday? 

6. What year were you born? 

7. What is your address here? 

8. What kind of place is this? 

9. Do you have a family? 

10. How often do you leave here (where you live) to go see your family? 

11. How often does anybody in your family come to see you? 

12. Are you usually. -happy or sad? 

13. Are you allowed to go on dates here? 

14. (If no) Why not? 

15. How many friends do you have? 

16. What do. you and your friends usually do together? 
(Probe: "What else?" until client can think of no more.) 

17. What vdo you usually do for fun when you are by yourself? 

18. Are you usually b^ yourself or with other people? 
Tell me about the most fun thing you did lasc week. 

19. What did you do? 

20. Who was there? 

21. When did it happen? 

22. Where did it happen? 



23. 



Why did you do that? 



24. 



Who decides what chores you do? 



25. 



Who decides how you spend your money? 



26a. 



Do you watch TV? 



b. (If yes) Do you watch TV alot, some, or not much? 

27. How much do you listen to the radio, or record player: A lot, some, not 
much, never? 

28. How much do you read books, magazines, or newspapers? 
29a. Do you go out to the movies? 

b. (If yes) Do you go to the movies not much, some, a lot? 

30. How often do you go out to eat? 

31. How often do you go to church? 
32a. Do you go to stores? 

b. (If yes) Do you go to stores not much, some, or a lot? 

33. Do you dcte? 

34. (If no) Why not? 

35. Do y. . play any sports, like baseball or footba.ll? 

36. (If yes to 'Vl'^y ^spor-ts") Which ones? 

37. Do you play any games indoors? 

38. (If yes to "indoor games") Which ones? 



39. Do you do any arts and crafts? 

40. (If yes) Which ones? 

41. Is anybody teaching you arts and crafts now? 

42. Are you usually happy? 

43. How many people sleep in your bedroom? 

44a. Do people here yell at youyor say mean things? 

b. Do people here take things away from you? 
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c. Do people here hit you? 

d. Do people here make you .do things you don't want to do? 

e. Do people here bother you when you want to be alone? 

f . Do people here make fun oif you? 

g. Do people here do. anything else that bothers you? 

h. (If yes to "anything else that bothers you/') What? 
45a. Do people here help you when you want help? 

b. IDo people here teach you things you want, to learn? 

c. Do people here give you presents? 

d. Do people here hug you? 

e. Do people here talk to you when you want to talk? 

/ 

f . Do people here say nice things to you? 

g. Do people here do anything else tha^ is nice? 

h. (If yes to "anything else nice") what? 

46. Are you usually with other people,? 

Now I want to ask you some questions about the rules here: 

47. Is it against the rules here to st^y up late at night? 

48. Is it against the rules here to hit people? 

49.. Is it against the rules here to^call people ugly names? 

50. Is it against the rules here to leave here without asking? 

51. Are you usually sad? 

52. Did you* ever get punished here? 

53. (If yes) What did you get punished for? 

54. (if yes to "ever get punished") How were you punished? 

55. -Who makes the rules here? 

56. Do you help make any rules here? 

57. Which picture.^hows how you fetj. about living here. Point to the picture. 
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Why do you feel that way? ^ 

How do you li^e the food here? Not at all, not much, some, a lot. 



60. 



Why do you felel that way? 



61. How do you like the people here? A lot, some, not much, not at all. 

62. Why do you feel that way? 

63. Did you ever live someplace else? 

64. Did you live in another place last month? 

65. Did you live in another place last year? 

66. Do you want to live someplace else? 

67. (If yes) Why? 

68. (If no) Why no? 

69. Are you usually with other people or by yourself? ^ 

70. Do you have any problems? 

71a. Do you usually have problems finding men/women friends? (same sex) 

b. Do you usually have problems finding men/women friends? (opposite sex) 

c. Do iyOU usually have problems knowing what to do when you get sick? 
' d. Do^yoL usually have problems with anything else? 

(If yes) What else? 

72. Who do you talk to about your problems? 
(Probe: Who is that exactly?) 

13: Who helps you the most when you need help? 
(Probe: Who is that exactly?) 

74. If you had one wish, what would you wish for? 

75. Why do you wish that? 

76. DO you know how to read and write? 

77. Is anybody teaching you about reading and writing now? 

78. Do you know how to cook? 

79. Is anybody teaching you about cooking now? 
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80. (If yes) Who 'is teaching you about cooking? 
(Probe: Where are you learning abouj: cooking?) 

81. Do you know how to keep house? 

82. Is anybody teaching you about keeping \house now? 

83. Do you set the table? (show picture) 

84. Do you do dishes? (show picture) 

85. Do you do laundry? ■ (show picture) 

86. Do you clean the floor? (show picture) 

87. Do you dust furniture? (show picture) 

88. Do you make beds? (show picture) 

89. Do you pick up stuff around the house/dorm? (^se proper term) 

90. Do you cook on the stove? (show picture) 

91. Do you make sandwiches? l ( show picture) 

92. Do you take out the trash? (show picture) 

93. Who decides what chores to do? 

94. Are you usually sad or happy? 

95. Do you get enough food to; eat? 

96. Do you go to school or take classes? 

97. (If yes) "What are they teaching you there?" 
(Probe: "What else?" Until the client can think of no m\)re.) 

98 « Do you know how to get a job?; 

99. Is anybody teaching you about getting jobs now? 

100. (If yes) Who is teaching you about getting jobs? 
(Probe: Where are you learning\about getting jobs?) 

101. Has anybody from vocational rehabilitation ever tried to help y?ju? 

102. (If yes) How did they help you? 

103. Are you usually by yourself? \ 

104. Do you have a job you get paid for? 
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105. (If no) : Have you ever had a job? 

106. Do you want to have a job? 

107. (If yes to "Do you have a job you get paid for?): Where do you work? 

108. (If yes to "have a job") : What exactly do you do at your job? 

109. (If yes to "have a job") : Do you like your job? 

110. (If yes to "have a job") : Do you want to keep your job? 

111. Did you have a job last month? 

112. Did you have a job last year? 

113. Do you know how to count money? 

114. Is anybody teaching you about money now? >• 

115. (If yes) Who is teaching you about money? 
(Probe: Where are you learning about money?) 

116. Where do you get your money? 

117. Do you get SSI? 

118. Do you decide how to spend your money? 

119. Could you buy some candy with $1.00? 

120. Could you buy a new bicycle with $1.00? 

121. Could you buy a new TV with $10.00? 

122. Could you buy a record album with $10.00? 
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Accessing Mentally Retarded Persons 
for Interviewing Purposes 



In order for any survey of a sample of the mentally retarded population 
to be carried out, attention to the regulations and guidelines governing par- 
ticipation.in research projects is necessary. As indicated in Chapters 3 and 
4, the procedures required in the various sites used in our interviewing pro- 
ject were different from one 'another. An interviewing project of this type 
depends on the cooperation of agencies and facilities to which mentally re- 
'tarded persons are known. It is entirely within the discretion of those agen- 
cies and facilities to' develop their own rules about access to clients, or in- 
deed to prohibit certain types of research or any research at all. Nonetheless 
there are certain guidelines that have gained common acceptance in practice, 
either because they are mandated or because tTiey .have been strongly advocated 
by key organizations. As part of .our project, we solicited information about 
such regulations and guidelines through a letter sent to a variety of public 
and private organizations and agencies involved with the developmen tally dis- 
abled. The_Jeri6wing summary is intended to suggest certain key procedures 
and issues which must be borne in mind. 

Assuming that a study has been planned and an intended population has 
been identified, the first step necessary is to obtain ' approval of the re- 
search proposal from appropriate review bodies constituted for that purpose. 
Institutions which receive federal research money are required to have insti- 
tutional human uses committees to screen all proposals and insure that the 
rights of participants are adequately protected. Although such committees 
were originally established to review projects to be funded by the Department 



232 



of Health, Education, and Welfare, ini|:reasingly they are screening the whole 
range of research projects involving human subjects, whether or not they 

are funded externally. \ 

1 • 

Institutional human uses committers are primarily concerned with the 
protection of confidentiality of infonAation, the secural of informed consent, 

■ . \ ' 

and the protection of subjects from phyjSical or psychological risk. The HHS 
regulations regarding the Protection of | Human Subjects (Titlel 45, Part 46, 

; 1 

revised April 1, 1977) are perhaps the most commonly followed regulations in 
the country because they must be adhered to if institutions are to receive 
HHS funding for research. In addition, studies of national scope require 
review of materials by the Office of Management and Budget. In our own project, 
for example, such a review would have been required if we had actually conducted 
a national survey, but was not necessary since we restricted our studies to 
Arkansas and Texas. 

Where the cooperation of axfacility or agency is required, as it inevi- 
tably would be in survey research with retarded persons, these agencies too 
have institutional review committees that must approve proposed research. Of- 
ten more than one body is involved. In Texas, for example, a research project 
to.be conducted at a state school for the mentally retarded must first be ap- 
proved by a local committee including representation from the community. It 
is then forwarded to a state level committee constituted by the state Depart- 
ment of Mental Health and Mental Retardation for approval. In the case of 
Texas, then, any research to be conducted by university staff involving resi- 
dents of the local state school would be approved by thi-ee bodies: the univer- 
sity human uses committee, and the local and state MHMR review bodies. In the 
case of interviewing studies, it is highly unlikely that the project would be 
judged to pQace subjects at risk, assuming that appropriate precautions are 

taken to insure that information provided by interviev;ees remains confidential 
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and that informed consent for participation is obtained. Nonetheless, the 
researcher must recognize that the process of obtaining approval can be 
very time consuming, taking as long as six months or so in ouiNexperience 
locally. Increasingly, too, local agencies and facilities are becoming 
hesitant to permit research which has no potential benefit for participants 
or for the facility staff. The benefits of participating in an interviewing 
study are remote (e.g., contributing to information which can be used to 
improve mental retardation services in the future) . 

r 

Several bodies besides DHEW have developed guidelines for research which 
may influence local facilities and agencies and of which the mental retarda- 
tion researcher should be aware. The American Association on Mental Deficiency 
Legislative and Social Issues Committee, for example, has developed a Consent 
Handbook (AAMD, 1977) . The AT^^MD position is that consent is not required for 
the use of aggregate data (for example, from facility statistical reports) ^ 
but would be required where individual subjects are involved unless the pro- 
ject involved anonymous interviewing about issues which are not personal in 
nature, in which case the subject's consent can be considered implied by his 
or her participation. Formal or explicit consent is recommended when the in- 
_ formation sought mighc identify tlie respondent or anyone else, when it is 
likely that unsought information will accompany the release of sought infor- 
mation, and when the information is protected under the First Amendment. 
While the AAMD generally advocates compliance with DHEW guidelines for in- 
formed consents the AAMD handbook is quite helpful in its explanation of consent 
as having three elements: capacity, information, and voluntariness. From 
the AAMD viewpoint, while DHEW emphasized information, there is a need in re- 
search with retarded persons for equal emphasis on determining the subject's 
capacity to understand wnat participation -involves and insuring that the con- 
sent provided is indeed voluntary. These issues are certainly less important 
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in an interviewing project that involves no risk than/ they are in biochemi- 
cal pr eyperimerital research, but the AAMD, quite rightly , emphasizes that 
special procedures might be necessary in view of the cognitive limitations ' 
of mentally retarded persons. Clark, Kivitz, and Rosen (1978) have dis- 
cussed in detail the tendency of mentally retarded persons to comply with 
various requests imprude'ntly (for example, to sign petitions without reading 
them or take pills without finding out whether they are harmful). Unfor- 
tunately, there ar^' limits to the ways in which an interviewing study can be 
explained to potential interviewees. The soundest procedure is to give a 
very simple and concrete description, encourage the potential participant to 
ask questions, and carefully, perhaps by asking the question in a number of 
different ways, insure that the individual indeed wants to participate and is 
not simply complying without understanding. When a subject is a young child 
or has been declared incompetent, a substitute decision-maker (e.g., parent 
or guardian) can make the decision on the subject's behalf. 

It should be noted that several bodies have developed positions in 
response to past exploitation of certain vulnerable classes of subjects, 
especially residents of institutions. The National Commission for the Protec- 
tion of Human Subjects of Biomedical and Behavioral Research has stated that 
such groups should no longer be overburdened by the demands of research and 
underbenef ited by its results. Other groups, for example the American Bar ' 
Association , have taken the basic position that if research uses mentally 
retarded subjects it must relate to mental retardation, thereby taking a 
stand against using captive groups to cojnduct research which might be done 
with any number of subject pools. A DHEW proposal of 1974 sought jo limit 
research with mentally retarded subjects to topics .most likely to benefit 
the mentally retarded, and the American Nurses Associate "n guidelines concur. 

The Joint "Commission for the Accreditation of Hospitals (JCAH) rec^pmmends 
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that research in institutions be explained to the staff and that the re- 
search results be implemented in the institution. In the most extreme case, 
the Willowbrook consent judgement, a court ruled that, given a history of 
abuses within the facility, no research upon members of the plantiff class 
would be permitted thereafter. This increased emphasis upon direct benefit 
evolves from an awareness of the coercive nature of institutions and the im- 
paired decision-making capacities of the mentally retarded people who T:e side 
in them. 

All documents on the use of human subjects in research emphasize the 
importance of maintaining a research participant's confidentiality. In 
particular, the problems of safeguarding against obtaining unsought informa- 
tion and prompting unanticipated outcomes are warned against. The confiden- 
tiality issue first crops up in identifying subjects. An interviewing study 
would require that agen^^ies of facilities participate in identifying current 
and/or former clients. \ Although the regulations do not appear to be standard 

\ \ 

in this area, a hard-line position would be that only the agency or facility 
could make initial contact with potential subjects, and would then release 
to researchers names and other requested information for those who provide 
informed consent to participate. Indeed this was the procedure required in 
one of our sites. However, other agencies or facilities may be willing to 
give researchers access to client rosters and files on the condition that 
they use the information only to solicit consent for participation in the 
research. It is likely that agencies and facilities will become more re- 
luctant to allow this access in the future. Whatever the regulations in a 
given setting, the researcher does have a clear responsibility to keep the 
names of potential or actual participants strictly confidential. 
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Interviewing studies may give rise to unanticipated problems of 

confidentiality. For example, the APA (1973) research participation guide- 
lines cite the example of a study of deinstitutionalized mentally retarded 
persons. Risks may arise if such a study involves contacts with neighbors, 
relatives, and employers, who may not have previously realized that the 
subject was mentally retarded or institutionalized. The precaution recom- 
mended by the APA is to restrict interviews to staff who already know the 
subject's status and to obtain permission from the subject if anyone besides 
the subject ^is to be interviewed. There is still the risk, however, that a 
mentally retarded person might not fully understand the implications of 
having hij or her employer or neighbors' interviewed. 

Another issue, also described in the APA document, arises when the re- 
searcher happens to acquire sensitive information, often information unre- 
lated to the research itself. Examples especially relevant ^-O interview 
studies include the discovery that a research subject abuses drugs, carries 
a weapon, is suicidal, or is engaging in unhealthy or destructive behavior 
of some sort. In such cases, researchers face the options of disclosing the 
information and thereby violating the subject's confidentiality or not dis- 
closing it and possibly being responsible, in a moral if not legal sense, 
for subsequent'-'hairm to the subject or others. Responsibilities of the re- 
searcher to disclose or not to disclose vary from state to state, and quite 
obviously the ethical issues are complex. One potential solution is to in- 
clude in informed consent statements an indication that the researchers might 
violate confidentiality if they feel a subject's welfare is endangered. In 
our own Center's research, a less severe but still difficult problem has at 
times arisen. Deinstitutionalized persons who have been interviewed have on 
occasion had transparent needs for help from service deliverers or have actually 

asked the interviewer to obtain help. Here, while it has been our policy to 
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comply with requests by referring the individual to appropriate local agen- 
cies, the research process , itself has introduced a change in the life being 
studied. Otherwise, confidentiality has been strictly i.aintained. Indeed, . 
clients had to be carefully convinced that participating would in no way 
jeopardize their community placements. 

Of course researchers always have a responsibility to protect the in- 
formation they collect and avoid giving it to other parties unless subjects* 
explicit permission has been obtained. Subjects must not be identified or 
identifiable in research reports or other dissemination vehicles unless they 
have given specific consent. 

In summary, our conclusion is that regulations and guidelines for re- 
search do not pose insurmountable problems in an interviewing study, but 
are problematic. If a ne^tional study were to be conducted, problems would 
be compounded. Obtaining permission to conduct the research from a variety 
of agencies and facilities in a variety of settings is likely to be time- 
consuming and costly and certainly must be considered in project funding and 
staffing. in an interviewing study, risk is unlikely to be an obstacle to 
approval, but in some settings questions may be raised about the likely bene- 
fit to participants and participating facilities, and in all probability the 
researchers will have obligations to provide reports to participating facili- 
ties. The most difficult problem, we believe, is identifying eligible sub- 
jects initially. Where explanation letters and consent forms must be mailed 
by various facilities or agencies before names can be released to researchers 
one may have difficulty obtaining agency cooperation because of the time in- 
volved lin mailings and a low rate of returns is likely, making the representa 
tiveness of the final sample suspect. Efforts to obtain consent which involve 
personal contact, by phone or better yet face-to-face, are likely to yield 
far higher rates of return, but may still be prohibitively time-consuming 
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from an agency perspective if agency staff must make the contacts. The, 
process is easiest if clief^ts are physically located in a facility, more / 
difficult if they only visit an office, and most difficult if former cli- 
ents are to be included in the sample, for then the initial task of loca- 
ting them may prove challenging and costly. By comparison, Liiu ta: j < 
obtaining consent and maintaining confidentiality once the project is 
underway are quite manageable , 

The following materials may prove useful to researchers planning sur- 
vey research, or any type of research, with mentally 'retarded populations:, 

American Association on Mental Deficiency: Consent ^Handbook , Washington, 
D,C,: AAMD, 1977 (Also statement on the use of human subjects for 
research) , 

American Psychological Association, Ethical principles in the conduct of 
research with human participants, Washington, D,C.: APA, 1973, 

National Association for Retarded Citizens: Mental retardation research; 
Guidelines for the use of behavior procedures in state programs for 
retarded persons , and Guidelines for biomedical and pharmacological 
research procedures and the protection of huirian subjects in residential 
facilities for, mentally retarded persons. 

National Commission for the Protection, of Human Subjects of Biomedical and 
Behavioral Research: Protection of human subjects; Disclosure of re- 
search information under the Freedom of Information Act; and Research 
with institutionalized mentally infirm subjects; Recommendations, 

B,8 



23j 



Department of Health, Education, and Welfare: A variety of reports and 
regulations, especially Code of Federal Regulations, Public Welfare 
45CFR 46, Protection of Human Subjects, 1977, revised edition. 



/ 
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Texas Tech University 



Rejearch and Trainrng Center in McnUl ReUrdah'^'^ ' (806) 742-3131 
Training Division / (606) 742-29/0 



Dear Parent: 

The President's Conwjittee on Mental Retardation** want 
the best ways to get intormation from and about retarded pe .e. 
have asked the Research and Training Center in- Mental Reta: .rior. 
Texas Tech to find this out for them. The Research and T: -ning ^ 
would like Lubbock students to help with this project. We would like 
to talk to some students to see if they can understand and respond to 
the questions we have developed, and we must have your permission before 
we can do this. 

The Research and Training Center will randomly pick about 60 students 
from those whose parents give their consent. . Then, two trained interviewers 
will spend about 15 minutes during the school day talking to each student 
who was chosen. This is not a test? it is only an interview. The interview 
will not change your child's school programs or placement in any way. 

The students will be asked some t, '^stions like "What is your birthday?" 
to see if they can give accurate inf om.ation about themselves. They will 
be asked about their feelings about themselves, others, and their situations. 
For example, one question will be "Are you usually happy or sad?" They will 
also be asked to tell about activities they are involved in like sports, 
indoor games, arts and crafts, and chores. Som*j of the questions ask about 
services they are getting and people who help them. (For example: "Is any- 
body teaching you to read now?") Finally, there will be some questions about 
rules in the home and school and decision making. (These will be questions 
like: "Are you ever allowed to go places by yourself without someone taking 
you?") 

All of this is conpletely anonymous. Everything your child tells the 
interviewers will be secret. We will not even tell anyone at the schobl 
w^hat his/hef answers are. In fact, the Research and Training Center is 
not really interested in what your child's answers are; we mainly want to 
know how well your child understands our questions. Also, if there are 
questions your child does not want to answer, he/she does not have to. 
If the interviewers are told or get the feeling that a student does not 
want to finish the interview, they will stop. 

There may be some students who will not be able to connnunicate very well 
or at all. We still need permission for them to be part of the study because 
one thing the presiden'.'s Committee wants to know is how many children cannot 
respond at all to the questions. 

**The President's Committee on Mental Retardation is a committee of 
people appointed by the President of the United States who help the presi- 
dent and Congress make policies and laws for retarded citizens of our coun- 
try. 
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Besides the student interviews, the Research and Training Center would 
like to spend about 20 ihinutes interviewiiig a parent or someone else who 
lives with the student and knows him/her- v :y well. We will be asking you 
questions about your child to find out i'f the answers your child gave were 
accurate. For example, if a student r-.-wered that he sets the table "a 
lot, Would the parent also sav that th'..j,r. child sets the table "a lot," 
or would he/she say "not. ni4ch?" For this fjarent interview, the interviewers 
will contact you before they come to your home or any place you would like at 
a time that would be best for you; even nights or weekends. if necessary. 
There is a place for your phone number and "address on the consent form. 
Please fill this in so they can contact you\to schedule the parent interview. 

Finally* the Research and Training Centfer would like to have your per- 
mission to use some basic information about ^our child from his/her school 
records. We would like the school to tell ua your child's bir'.hday, IQ 
score* and if he/she has other handicaps. 

If you and your child help with this project, the Research and Training 
Center will be glad to give you a copy of the: results . While this report will 
not tell you anything about your child individually, " will give you some 
information about how children feel about their lives and what their strengths 
and weaknesses in communication ski.Ms are as: a group. 

If you have any questions about this pro'ject, please feel free to, call 
Carol Schoenrock at the Research and Training- Center. Her phone number is 
742-3134 or 742-3131, extension #38, She will^ be happy to answer any ques- 
tions you might have about this study. ■ 

\ / ■ 

When you have decided whether or not you Vant your ch^ld to help with 
this project, please fill out the attached fonij, showin^that ypu either 
give your permission or do not give your permission f<5r your child to partici- 
pate. Please give this form to your child to take back to school tomorrow. 
Thank you for your consideration. 

Sincerely, 



Carol K. Sigelman, Ph.D. 
Director of Research 
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Texas Tech University 



Research and Training Center in Mental Retardation / (606) 742-3131. 
Training Division / (806) 742-2970 

August 7, 1978 
Dear Advocate: 

The president's Coiranittee on Mental-Retardation has asked the Research and 
Training Center to undertake a project for them. They want us to determine 
tne feasibility of establishing a national polling system that would enable 
them to quickly gain inforr.^tion about the needs, circumstances and atti- / 
tudes of retarded citizens. By getting input from mentally retarded / 
consumers themselves, they hope to make national policy formation more 
sensitive to the actual rteeds and opinions of these consumers. 

As part of this study, we are presently trying to determine the best meth- 
ods for obtaining certain types of informa'iion from ard about retarded per- 
sons; in this case, adults living in the community. The Citizen Advocacy 
office in Austin is helping us to establish a sample of such persons to 
interview, and your name was given to us because you are an advocate of . . 
someone we want to include in that sample. 

During the last few weeks in August > a trained interviewer from the Center 
will conduct personal interviews with the proteges. These intervievs should 
take approximately 30 minutes each and will focus on residential living 
circumstances and on services received. We will address ourselves to the 
extent of the proteges' knowledge about these areas as well as their atti- 
tudes toward them. In addition to this, a telephone interview will be 
conducted with each person's advocate. (A personal interview can be ar- 
ranged if the advocate desires.). This l5-minute interview will cover the 
same topics found in the protege intervi'jw, allowing us not only to deter- 
mine the accuracy of the protege's responses to different types of ques- 
tions, but also to obtain a different perspective. Of course, both inter- 
views \wi 11 be completely anonymous and confidential. 
\ 

Before we can interview you or your protege, we need written, informed con- 
sent frq^m both of you. Enclosed with this letter is an explanation of our 
project Which we ask that you read to and discuss with your protege before 
he/she signs the consent form (also enclosed.) If your protege is not 
capable of giving informed consent, please discuss this with his/her gueurd- 
ian and l4t him/her sign the concent form in the space labeled GUARDIAN 
CONSENT. Regardless of who signs the consent form, we would appreciate 
your informing the guardian (or other person acting as caretaiker unless the 
protege lives independently) that we will be contacting them or your pro- 
tege to arrange a time and place to conduct the interview. 

We have enclosed a self -addressed, stamped envelope for your- convenience in 
returning the consent form. This needs to be returned whether both of you 
give your consent or not. The President's Committee also wants information 
about refusal rates. However, we hope you will help with this project, for 
we can all benefit from techniques to help retaurded citizens express their 
points of view. If you or amyone else have any questions, I would be more 
than happy to accept a collect call at (806) 742-3131. 

Sincerely, 



Csurol Jean Schoenrock 
Project Coordinator 
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EXPLANATION OF R & T CENTER PROJECT 
(To be read to the protege by the advocate) 



Some people who work at Texas Tech University in Lubbock want to come 
and talk to lots of people who live in Austin. If you want to, they 
would like to talk to you for about 10 minutes. 

They want to ask you questions about where you live, what places you • 
go to get help or learn things, who is teaching you things, what things 
you and I do together, and how you feel about where you live and the 
help you get. 

They want to find out the best way to ask people these questions, and 
they want to find out what things are like for people who live in Austin. 

No matter what you tell them it will not make- cJ ay changes in your life. 
\ They are not trying to test or evaluate you; they want to test their 
questions to see if the questions are good enough to use with people 
all over the United States. 

You do not have to talk to them at all if you do not want to. If you 
do talk to them, everything you oay will be a sqcret. Nobody else 
except the person you talk to will knew exactly what you say. They won't 
even tell me what you say. If you talk to them and there is a question 
you don't want to answer, you don't have to; you can just tell them you 
don't want to answer that. 

They also want to taTc to me and ask me some questions about you and the 
things I do with you and for you. Everything I say will be a secret, 
too. They won't tell anybody else, even you, what I tell them. 

Before they interview you or me, they want to look at our records in the 
Citizen Advocacy office to get some basic information. 

They will send a report to the Citizen Advocacy office and to some other 
people -about the interviews they do with everyone here in Auatin. But 
rt will not say anything about you or me. It will say things like: 
"Such and such was a good question; most people understood it," or 
"This was not a good question to ask; we need to find a better way to 
ask .it.*' Also, they will say things in the report like: "Most pro- 
teges said their advocate did such and such with them," and "Many of 
the proteges would like to learn such and such." 

Before they interview you, they will call or come by your house to set 
)up a time to do the interview. That way, you can tell them the best 
I time and place for you to do it s". they won't bother you when you are 
I going to be busy. 

If you have any questions about this project that I can't answer, we can 
call the people at Texas Tech and ask them before you decide if you want 
to help them. 



11. When you decide, you need to sign the consent form and check one of the 
Clanks telling them that you do want to help or that you do not want to 
help. Then I will send it back to them, if you check that you do want to 
help, they will get in touch with you in a couple of weeks to doThe inter 
View. 
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ADVOCATE'S NAME: 



PROTEGEES NAME: 



CONSENT FORM 

I understand the project that the Research and Training Center is doing. I un- 
derstand that I will be> interviewed and that what I sa^ ill be secret and anon- 
ymous. I know that I do not have to answer any questions I do not want to an- 
swer and that this interview will not mean any changes in my situation. I also 
understand that the interviewer will use information from my records at the Citi- 
zen Advocacy office. 

PROTEGE ! 

(check one) 

.YES, I do want to talk to the people fiom Texas Tech. 

No, I do not want to. talk to the people from Texas Tech. 

PROTEGE'S SIGNATURE: __DATE : 

GUARDIAN CONSENT: DATE: 



ADVOCATE : 

(check one) 

YES, I do wan-c to talk to the people from Texas Tech, 

No, I do not want to talk to the .people from Texas Tech. 

ADVOCATE'S SIGNATURE DATE: 

xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx 

IF YOU HAVE AGREED TO HELP WITH THIS PROJECT, PLEASE COMPUiTE THE FOLLOWING INFOR- 
MATION: PROTEGE: 

Age:_^ [ Sex: Telephone nuniber: 

Address (include town & zip code) : . 



If we should contact someone other than the protege to set up the ap- 
pointment for the interview, please give us their name (plus phone and 
address if different from the protege's) : 



ADVOCATE : 

Tl aone number: 



Acidress (include town & zip code) 



If there is any time that is generally better than another for us to 
contact you or your protege, please indicate below: 

Advocate: Protege: 

C.6 2iy 



SAMPLE INTERVIEW SCHEDULE PAGE 
The following page is a reproduction of the first page of the 
interview schedule used' with community children. Regarding the method 
used to code answers, the notation refers to content and is where the 
respondent's answer is coded. Multiple C columns were used for open-ended 
questions in which several answers could be enumerated. The "R" variable 
for each question is the responsiveness code for that question. If the 
interviewee was unable to respond adequately to the question - for example 
if his answer was irrelevant - the R code would indicate that and the C code 
would designate missing data. This system made it easy to keypunch data 
directly from the interview schedule. 
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Coding System Used on Interview Schedules 

1. ARE YOU USUALLY HAPPY OR SAD? ^ 

Sad s= 1 Happy = 2 ^ HSl R:_ 

/ 

2. ARE YOU USUALLY WITH OTHER PEOPLE? ^/ WOP C:^ 
No = 1 Yes 7^ 2 

WOPl R: 



■ • ./ 

The blanks in the right-hand column permit keyp\inching direct- 
ly from the actual interview schedules- The respondent answered 
the first question "sad". The "C" in the right-hand coliamn indi- 
cates content coding, and the "sad" refi-*^nse is accordingly coded. 
1. The "R" indicates a responsiveness code; the 8 represents a 
minimally appropriate response. The numbers beneath these coding 
blanks indicate the correct column on the cards used in keypunching 




CLIENT INTERVIEW Client Name: 

Consumer Study 

Interviewer Name: 
Date of Interview: 

Research and Training Center 

in Mental Retardation Time Scheduled: 



Beginning Time: 

Texas 'Tech University 

Lubbock, Texas Ending Time: 



COMMTJNITY CHILDREN 



■•Incorrecr -»-l ---^^-"^Part"ial^ - 



a : 3 



STUDY 

CAJUD : 0 : 4 

3 4 

CASE ... 

5 6 T" 

INTNUM 



1. WHAT IS YOUR FULL NAME? 

(Probe: "What is your last name?" IntP- viewer may an- C : 

swer questions: no middle name needed, real name; ii 
not nickname, etc.) 

R ; 

Incorrect = 1 "Partial = 2 Correct = 3 



HOW DO YOU SPELL YOUR NAME? 

(Probe: "How do you spell your last/first name?") C : 

Int. may answer questions:' first and last name, i3 
not" middle name, real name, etc.) _ 

R 5 

-14-,-- 



3. PLEASE WRITE YOUR NAME : OR ME ON T.^^i 

('^58 ack of form. Int. may answ**:. ^s^^ons: first q . 

and last name, not middle name, real name, etc.) 

Incorrect - 1 Partial = 2 Correct = 3 ^ • 

u 
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CLIENT CONSENT AND INTRODUCTION 



HI, MY NAME IS . I'M TRYING TO FIND OUT THE 

BEST WAY TO ASK PEOPLE QUESTIONS. A LOT OF PEOPLE HERE IN LUBBOCK ARE GOING'' 
TO HELP ME, AND I'D LIKE FOR YOU TO HELP ME, TOO. YOU DON'T HAVE TO DO THIS 
IF YOU DON'T WANT TO, JUST LET ME TELL YOU WHAT I WANT YOU TO DO, AND THEN 
YOU CAN TELL ME IF YOU WANT TO HELP. I WANT TO ASK YOU SOME QUESTIONS ABOUT 
YOURSELF; MOSTLY ABOUT HOW YOU FEEL AND WHAT YOU DO AT HOME AND AT SCHOOL. 
IT WILL TAKE ABOUT 15 MINUTES. EVERYTHING YOU TELL ME WILL BE A SECRET. I 
WON'T TELL ANYBODY ELSE WHAT YOU SAY. IF YOU DON'T WANT TO ANSWER A QUESTION, 
YOU DON'T HAVE TO. JUST TELL ME YOU DON'T WANT TO ANSWER IT. THIS NOT A 
TEST. ANY ANSWER YOU GIVE ME IS OK. I'M JUST TRYING TO FIND OUT IF PEOPLE 
CAN UNDERSTAND MY QUESTIONS. DO YOU WANT TO HELP ME DO THIS? 

(If no, stop here. Thank client for taking time to listen.) 
(If yes, continue with explanation below.) 

OK, GOOD. BEFORE WE START, LET ME TELL YOU S0m6 THINGS. SOME OF THE QUESTIONS 
MAY SOUND SILLY TO YOU, BUT I HAVE TO ASK THEM ALL, EVEN IF I ALREADY KNOW 
THE ANSWER, I STILL HAVE TO ASK IT. I'M GOING TO BE ASKING A LOT OF THE QUES- 
TIONS MORE THAN ONCE. THAT WON'T MEAN THAT YOU GAVE ME THE WRONG AN5WER THE 
"FIRST'TTME . " THE ONtY "I^ASON" "r*LL"^^^^ ONCE IS" • 

TO FIND OTIT WHICH WAY OF ASKING THEM" IS THE BEST WAY. REMEMBER, TELL ME IF 
YOU DON ^ ANSWER A QUESTION I'" ^ ;U. 
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